Articles Feed

JavaScriptness.prototype = new Class(); From Classical to Prototypal

by: justin | July 6th, 2010 | 0 comments »

JavaScript is a very misunderstood language.

It has gotten a bad rap because most people that use it don't look for its brighter side. Instead they will often look online for some code to copy paste, or they will hack something together using tiny snippets and in-lines to get a drop down menu.

It is also a very conflicted language. By nature, JavaScript is a Prototypal language where nearly everything is an Object and there are no classes. It is also somewhat Functional since all the functions are First Class Functions. They are just more objects that can be passed into and returned from other functions.

The Classical Form

Lets take a look at a basic Classical implementation of a Square using JavaScript:

function Square (side) {
  this.side = side;
}
Square.prototype.area = function () {
  return this.side * this.side;
};
Square.prototype.perimeter = function () {
  return this.side * 4;
};

var mySquare = new Square (5);

You will notice that JavaScript even has a new key word in an attempt to match a Classical implementation. The function Square () defines a class with a side attribute. It also has two instance methods, area and perimeter. We can then create mySquare in almost the same way we might do it in Java. This is JavaScript though, and the new works differently in JS. It creates a new, empty object and then calls the Square() function (or the constructor) passing the empty object into this . The constructor then populates that object with the specified behavior before returning it. This is particularly scary, because if you forget to put new in front of the constructor, this will refer to the Global Object, as will mySquare (meaning any further changes made to mySquare would also be on the Square 'class'). This is why, if you want to use JS in the Classical way, you should always capitalize the first letter of a constructor.

Ok, so we have our Square, but say we want to create a square that holds a X or an O. Well, we would want to inherit the properties of our current Square 'Class' and then add to them. We might do something like this:

function ContainerSquare (side, contents) {
  this.superclass(side);
  this.contents = contents;
}

ContainerSquare.prototype = new Square();
ContainerSquare.prototype.superclass = Square;
ContainerSquare.prototype.constructor = ContainerSquare;
ContainerSquare.prototype.getContents = function () {
  return this.contents;
};

var myContainer = new ContainerSquare(6, "X");

We create a new 'Class' for the ContainerSquare which uses the Square's constructor to define the side attribute, along with the two instance methods. We then define our new contents attribute, along with an accessor method to go with it.

The ContainerSquare .prototype = new Square(); line sets up a Prototypal inheritance structure so that a ContainerSquare object will be linked to the proper prototype chain.

The prototype chain is the hierarchy of objects that a method or attribute call will traverse until it finds what it is looking for. In other words, when I call myContainer.size(), JavaScript will first check to see if the myContainer object has that function. If not, it will proceed to check the object's prototype, which is the ContainerSquare. Since ContainerSquare doesn't have the function, next in line is ContainerSquare's prototype, which is just Square. Square does in fact have a size() function, and since javascript will use the first function it finds, it will use Square's size() function.

Thus, by setting ContainerSquare's prototype to Square, we inherit any of the attributes or methods of Square.

Although this all works, and we have successfully mashed a Prototypal language into a Classical scheme, we should try this in the way JavaScript wants to be used.

The Prototypal Form

We can start with the Square. There are few pretty simple ways to achieve the same functionality using regular objects and their prototypes, and we will look at two of them. The first will create a square object that we can use. Then if we need more squares, we can make a copy of that object using its prototype. The second will create a squareMaker function, which can then be used to pop out new squares.

var firstSquare = {
  side: 5,
  area: function () {
    return this.side * this.side;
  },
  perimeter: function () {
    return this.side * 4;
  }
};

Here, we already have a new and usable square at our disposal. We don't need to call a constructor and define any types. We can just take an object and mold it into the form we want.

Say we want another square though. We can't just write something like var secondSquare = firstSquare; because JS passes objects by reference, and thus the second square would just point to the first square. What we can do, and this is a technique developed by Douglas Crockford, is make a copy of our object by calling a new constructor with a prototype that points to our object. This new contructor creates a new empty object and assigns the values of our old object to the new empty object. You can do this yourself, or you can use Crockford's technique as follows:

if (typeof Object.beget !== 'function') {
  Object.beget = function (o) {
    var F = function () {};
    F.prototype = o;
    return new F();
  }
}

var secondSquare = Object.beget(firstSquare);

secondSquare.side = 6;

You can see that we are actually defining an on the fly constructor F, using our object 'o' to define F's prototype (where 'o' is firstSquare). Then we create a new object using that constructor. This will give us a new copy of our object, with all its attributes and functions. If we now called secondSquare.area(); we would get 36.

If you wanted to then make a ContainerSquare, you could simply add a contents attribute to the second square, and then make copies of the secondSquare if you need more Containers. Keep in mind that since firstSquare is the prototype of secondSquare, if you were to add a contents attribute to firstSquare, you would then have that attribute on secondSquare; however, adding attributes to the secondSquare does not place them on the firstSquare.

Spawn More Protolords

The other way to get squares would be to make a squareMaker function. This function will return a new object with whatever attributes you define. I am also going to show you some closure so that the attributes are private, and only accessible through accessor methods.

var squareMaker = function(side) {
  return {
    getSide: function() {
      return side;
    },
    area: function () {
      return side * side;
    },
    perimeter: function () {
      return side * 4;
    }
  };
};

var anotherSquare = squareMaker(5);

You will notice that the return value of the squareMaker is almost exactly like how we defined our firstSquare object. We are just returning the definition of a square, and thats quite awesome. Another thing you will notice is the bit of closure. We pass a value into the squareMaker, but it isn't stored anywhere. It is held in the scope of the squareMaker function, allowing those internal methods to use it, but hiding it from the outside (unlike the side attribute defined in the firstSquare). This sort of closure is another wicked awesome tool you can use in JS. In a way, this use of closure is like defining private variables to a class.

To get the Container functionality we create a containerMaker using the squareMaker, and some more closure, to make a new square object. We then dynamically add a getContents method to provide access to our private contents attribute.

var containerMaker = function(side, contents) {
  var container = squareMaker(side);
  container.getContents = function () {
    return contents;
  };
  return container;
};

var anotherContainer = containerMaker(6, "O");

Conclusion

So we have now seen two ways to use some of the better parts of JavaScript to get the same functionality that we could get using the Classical scheme. We can also see some advantages in using JS in its natural form like: getting some closure (which can be exceedingly powerful), very dynamic objects ready to change in anyway you can think of, quickly defined objects which can be used and multiplied, and no need to predefine types. Pretty sweet right?

I will admit one slight inefficiency with the two solutions I showed you (although there are ways around this). Using the squareMaker, or making copies of the objects will make full copies, including the function objects defined inside. If you use the Classical scheme I showed, you are defining functions on the prototype rather than the object itself, thus there will only be one copy of the function. You can, of course, do something similar in the JS scheme, but the optimization doesn't count for all that much in most cases (unless you are doing mobile development where you want to save everything you can!).

JavaScript isn't a Classical language, but instead a Prototypal language that is powerful enough to mimic a Classical language without breaking a sweat. There is much to be gained by recognizing this fact and changing your mindset to work with it. Trying to fit a square into a circular hole will just get you stuck. All languages are unique with a variety of their own advantages, and as good developers we should recognize these differences and reap all the benefits they have to offer.

MMEmail: My First Clojure Open Source Contribution

by: micah | April 21st, 2010 | 0 comments »

While working on website for my sister, I was perturbed that there were no libraries on Clojars for sending email. I was using Leiningen which will download and include all your dependancies for you (so sweet!). Yet, with the library I found, I had to manually download all the required jars. Call me lazy but being forced to manually download dependancies seems like cruel and unusual punishment these days.

So I solved the problem:

MMEmail

Simple Clojure library for sending email, with just one jar!

Installation

Leiningan:

1. (defproject your-project "0.0.0"
2. :dependencies [[mmemail "1.0.0"]]

Jar File

…can be downloaded at http://clojars.org/repo/mmemail/mmemail/1.0.0/mmemail-1.0.0.jar

Usage:

Include the library

1. (use 'mmemail.core)

The Easy Way

mmemail.core is the only include you need. It contains only 2 methods, the fist being send-email. It takes a map that includes all the configuration and email parameters. That’s it.

1. (send-email {:host "smtp.gmail.com"
2. :port 465
3. :ssl true
4. :user "road@gmail.com"
5. :password "runner"
6. :to "joe@acme.com"
7. :subject "Greetings"
8. :body "Meep Meep!"})

Most of the parameters are required, but based on your server configuration you might get away without these:

• :ssl
• :password
• :subject

It also accepts the following optional parameters:

• :from (defaults to :user)
• :cc
• :bcc

The recipient parameters (:to, :cc, :bcc) may be a string, for one recipient, or a sequence of strings, for multiple recipients.

The Convenient Way

It can be annoying to pass such a big map into the function, and typically you’ll want to get all the server configuration out of the way. This is where the second function of mmemail.core (create-mailer) comes into play.

1. (def my_mailer (create-mailer {:host "smtp.gmail.com"
2. :port 465
3. :ssl true
4. :user "road@gmail.com"
5. :password "runner"}))

create-mailer will return a new function with all the configuration baked in. So in the future, you can send email like so:

1. (my_mailer {:to "joe@acme.com"
2. :subject "Greetings"
3. :body "Meep Meep!"})

The create-mailer accepts email parameters that will be used as defaults when the generated function is called.

1. (def my_mailer (create-mailer {:subject "Greetings"
2. :body "Meep Meep!"
3. :host "smtp.gmail.com"
4. :port 465
5. :ssl true
6. :user "road@gmail.com"
7. :password "runner"}))
8. (my_mailer {:to "joe@acme.com"})

License

Copyright 2010 Micah Martin. All Rights Reserved. MMEmail and all included source files are distributed under terms of the GNU LGPL.

TDD in education, they're doing it wrong

by: eric | April 9th, 2010 | 4 comments »

This blog post originally appeared on a blog I maintain for my current game development class. Generally speaking my posts over there are pretty specific to the classes I’m taking, and aren’t appropriate for this blog. Recently however my instructor, who is an extremely smart and accomplished developer in the game development industry, described a common misconception about TDD that prompted me to rant on that blog. Since the subject is TDD I’m placing it here, edited to make more sense.

Recently in class our professor said something like this, “TDD says you should only write the simplest code that can possibly pass the test, and that leads to spaghetti code with a lof conditional logic.” Sadly this left me shaking my head. I’ve been doing TDD every single day for the last 3 years professionally, and intermittently for 2 years before that, and I can say for certain that TDD never caused me to write spaghetti code. Of course I’ve written spaghetti code, including some of the code I TDD’d for the previous class (I’m not real happy with my memory system) but that wasn’t caused by TDD. The crux of the professor’s argument seems to come from a mistaken understanding of TDD. He seems to have TDD as a three step process:

• Write the test
• Write the smallest amount of code to make the test pass
• Repeat

Unfortunately it’s missing a crucial step – refactor! Yes we initially write the simplest code that can possibly pass, but we don’t leave it that way. We write that simple code so we can refactor with a green bar, something that unfortunately my professor doesn’t understand.

When I write the minimal amount of code refactoring becomes easier, because I have a suite of passing tests. In school students and teachers throw around the term refactoring a lot, but are usually unaware that when Martin Fowler wrote the book on Refactoring, defining it for the software development community, he required unit tests. You can’t guarantee your behavior did not change unless you have tests around the behavior, which is why I hear so many students say things like “I’m almost done refactoring but the code doesn’t build yet.” Refactoring is meant to be small steps, in a definite process, so that at the end of each refactoring (such as extract method) I have the same behavior as before, but better code. Most people who refactor don’t have tests, take huge steps and eventually end up hacking and slashing something back together that is close to (but not the same) as their original behavior. That’s not refactoring, that’s hacking - and most students clearly don’t know the difference.

At this point the blog went into a description of how you’re supposed to do TDD, but if you’re reading this blog you probably already know how to do that. The point of moving this here is to point out that TDD is making inroads at the university level, but it has a long way to go. Those doing TDD are a distinct minority, those that are are doing it wrong, and they’re getting bad information. Furthermore let’s remember these are smart people. The professor for my class is frakking brilliant, and the students are amongst the best in the university, and yet in spite of that when they start their first job they’ll do so without writing unit tests. The good news is that smart people are trying out TDD, in an industry (games) that is notoriously slow to adopt software development methodologies, so we are winning. It’s just taking longer than we thought.

Stubbing :new Considered Harmful

by: eric | February 8th, 2010 | 1 comments »

Hi my name is Eric, and I have made a mess.

There I said it. I’m not proud of it, but I believe a couple very loosely related things:

• You only learn from your mistakes
• Everybody makes mistakes, a craftsman cleans them up.

It is in the process of cleaning up the mess that I’ve been trying to evaluate the messy code in the first place. How is the code messy? Was it Test-Driven? It was - or at least it attempted to be (we’ll get back to that). Was I lazy? Certainly not. Am I stupid? Crap I hope not. So how did we end up in this mess?

What’s the mess nimrod?

Recently Justin, our latest apprentice, wrote about attending a code review here. The feature he is referring to is part of one of our best client’s systems, where we integrated with several webservices, making them look seamless in our application. This was an extremely difficult problem, and our code bent in a dozen painful ways for a solution, and has since become rigid. Changes have stopped being done in the nice Test-Code-Refactor mode, and instead are being done by trying a change, trying the product, then backing out the change and making the test pass. While we identified several ways to improve the code, such as better naming and adding a facade between some of our interfaces, one stated goal was to “make the tests readable.” See the problem isn’t that it’s not tested - it’s that you can’t follow the tests at all. Let’s look at an example, modified to protect the guilty:

2. it "should call a webservice when its complete" do
3. @caller.should_receive(:call_webservice)
4. typedObj = mock("Obj", :no_follow_ups? => false, :null_object => true)
5. ProprietaryObject.stub!(:create_typed_obj).and_return(typedObj)
7. @page.stub!(:widget_id).and_return(@widget.id)
8. @page.stub!(:previous_page_id).and_return(nil)
9. @product.page_cache = [@page]
11. @widget.follow_ups({}, {:application_id => "123", :group => "group"})
12. end

This probably isn’t the worst example - if you look closely you can see that we’re testing that call_webservice needs to get called when ‘its complete.’ But what complete? I don’t see a stubbed object that returns complete. Also I have no idea what @page, @widget, @product or :createtypedobj are doing here. When I want to write the next test I have to guess at intent, and start playing with variables. What this is trying to test is that a webservice caller object receives call_webservice when the widget (an object on the screen) is called with certain variables. I’d tell you what those variables are, but I truly don’t know.

It doesn’t get better. Let’s look at where some of those variables are coming from. These are excerpts from a setup method that is too long.*

2. Context.stub!(:find_by_widget_and_application_form).and_return(@context)
3. Page.stub!(:new).and_return(@page)
4. ...
5. @caller = mock('caller', :call_webservice => nil)
6. WebserviceCaller.stub!(:new).and_return(@caller)
7. ...
8. @page_factory = mock_model(PageFactory, :acquire => @page)
9. PageFactory.stub!(:new).and_return(@page_factory)

So by my count we’ve stubbed new three times, two finders, and an acquire all of which inject yet another dependency into the code. Thus the premise of the article stubbing new considered harmful.

Is it really harmful?

Dependencies in static languages are generally more onerous than what we’ve got here, sometimes extremely painful although usually it can be accomplished with simple setters and constructors. In Ruby this isn’t the case - you can just stub new and create the object the way the user intended, thereby continuing to test. This can be a VERY BAD THING because it’s a moment where you need to pause. Do you need this dependency? Should you be wrapping dependencies? Do you need a facade or wrapper object for a couple dependencies? The difficulty of DI in other languages enforces that pause, but Ruby does not. It lets you stub anything, only to one day wonder why you have such a mess on your hands.

It’s important to remember that Dependency Inversion existed before Unit Testing, and isn’t just a Unit Testing technique. Indeed we often sell TDD as a way to make you have better designs, and yet here it failed me because I can so simply call new and continue testing. This is a mistake. To paraphrase Jurassic Park, just because you can easily stub new doesn’t mean you should. Would you create a constructor that took five objects? Then why would you stub :new (or finders, or factory methods) five times. Would you pass in an object to the constructor that’s only used in one method? No - that’s not cohesive. By stubbing :new you can slowly, and easily, introduce dependency after dependency creating an indecipherable mess despite writing tests. Worse you can do this easily, without suffering the traditional pain of tightly coupled code. It’s only later that you’ll feel the awful, horrible, indescribable pain. Trust me it’s bad.

Am I really advocating stopping stubbing :new? No not really. It makes perfect since to just call :new when you can, and in many cases factories are a way to get around the limitations of a static language. What I am saying is that each time you do stub!(:new) THINK! You have a dependency to manage, so manage it. It’s your job you know.

*This is also a smell. Allow me to quote “The Art of Unit Testing”, an excellent book on Unit Testing by Roy Osherove

Don’t use [SetUp] and [TearDown] to initialize or destroy objects that aren’t shared throughout the test class in all the tests, because it makes the tests less understandable. Someone reading your code won’t know which tests use the logic inside the setup method and which don’t.

Reflection on Hangman

by: micah | October 4th, 2009 | 7 comments »

Last week concluded the Hangman Ruby Sparring Tournament. The results are below.

Unlike the previous Battleship Tournament, I put the effort in to write a competitive hangman player (of course he was banned from competition). This made things interesting. By knowing precisely what it takes to create a player, I was able to precisely tune the scoring metrics. My goal with the scoring metrics was to evaluate the code, without actually looking at it myself, and rate it’s quality/cleanliness on a scale of 0 to 100, where 100 was asymptotically impossible to reach.

It is not easy to score well on ALL the metrics. Each metric will push your code in a different direction some of them are opposing. For example, to score well on the flog metrics, you have to break your code up in to lots of tiny methods. However, extracting methods increases the amount of code you have which brings down the simplicity score. To get an overall high score requires compromising between opposing metrics over and over again. Realistically, writing clean code requires the same type of compromising between coding principles. I am thoroughly impressed with the high scores that people were able to achieve in this tournament.

In the end, I believe that the highest scoring players do fit many of the criteria for “clean code”. Which is to say that to metrics used to evaluate the code are fairly complete. However, there is one gaping hole. The highest scoring solutions look like this:

1. def b
2. @r.fill(0)
3. c
4. @l = @l.sort_by { |x| @r[x] }
5. end

People discovered that by using short variable names and method names, they could considerably reduce the “simplicity” of their solution. For this tournament, the simplicity score measures the code mass (compress all the code and count the bytes). It’s a fair metrics because in general, the less code there is, the less code people have to read to understand it. I especially like the fact that comments hurt the score. Anyhow, clearly the code above is not good code.

What’s missing is a analysis tool to measure “Readability” of code. I imagine such a tool would be similar to flog and flay in that it parses your code and examines all names used for variables, methods, classes, etc. When it finds 1-letter names like above, it punishes. When it finds names containing english (or other languages) words, or derivatives of other names, it rewards. For example, it I had a class named “Player”, that’s an english word which is readable. Good. Now a class named “ThingAmaBob” is not english but that not necessarily bad. And you’d expect variables with names like “mythingamabob” or “thingama_bobs” which would be good. There are plenty of ways to expand on the idea. Such a tool would bring our repertoire of metric tools one step close to completion.

TDD and iPhone - NSTimer

by: eric | May 11th, 2009 | 3 comments »

TDD on the iPhone is a challenging experience, especially when you’ve been spoiled by Ruby like I have been, but it can be done. I have a tutorial in the works for getting started, but in the meantime I’ll be writing bits and pieces about my experience. The first was here and this is the second. Test cases are written using the Google Toolbox for Mac. This particular example is about the NSTimer.

It keeps going and going

The app to your right is a simple app for experimentation purposes that runs Conway’s Game of Life. It has two buttons: advance and start. Advance moves one generation ahead each time you press it, whereas Start keeps advancing generations until you touch Stop. To implement that we use an NSTimer object which is programmed to call advance on the game object. The design is simple, shown here:

The GameRunner creates an NSTimer, the NSTimer sends the message to Game. Great but how do we test it? The less I know about something the more I tend to test it, which means I tend to write smaller and smaller tests. Let’s look at my first few tests of the GameRunner object:

1. -(void) testSetsAndGetsTimeInterval {
2. runner.interval = 0.30;
4. STAssertEquals(0.30, runner.interval, nil);
5. }
7. -(void) testCreatesNSTimerObject {
8. [runner start];
10. STAssertNotNil(runner.timer, @"The timer was not created");
11. STAssertTrue([runner.timer isValid], nil);
12. }
14. -(void) testNSTimerFields {
15. runner.interval = (NSTimeInterval) 0.26;
17. [runner start];
19. STAssertEquals([runner.timer timeInterval], 0.26, nil);
20. }

We’ve got some basic tests of creation. At this point I was stumped. I couldn’t check the other parameters of the timer, because they weren’t public variables, so at this point the code that actually created the timer looked like this:

1. -(void) start {
2. timer = [[NSTimer timerWithTimeInterval: interval
3. target:nil
4. selector:nil
5. userInfo:nil
6. repeats:true] retain];
7. }

That doesn’t really help me does it? I have a timer with an interval, but it won’t call anything when it’s triggered. This is about the time I got to pair with Jake Scruggs during the Craftsman Swap, which he’s written about here. It went badly, somewhat embarrassingly for me since I’m supposed to be the expert, however we learn from failure. Let’s look at some of the steps Jake and I took that didn’t quite get us there.

The first pass on the test was simple. We needed a mock game, and it needed to have its advanceGeneration method called (Ed. note: I realize they aren’t called methods in Obj-C, to which I say: bite me.) each turn. Let’s try the brute force approach:

1. -(void) testTimerCallsGameAdvance {
2. MockGame *mockGame = [[MockGame alloc] init];
3. runner.game = mockGame;
5. [runner start];
6. sleep(.30);
8. STAssertTrue([mockGame advanceGenerationCalled], nil);
9. }

I’m writing the original tests from memory, so I make the occasional error here, but it doesn’t matter since we got this wrong. This test here seemed the most logical. We ran the test and it failed, yea! Then we updated the code:

1. -(void) start {
2. timer = [[NSTimer timerWithTimeInterval: interval
3. target:game
4. selector:@selector(advanceGeneration)
5. userInfo:nil
6. repeats:true] retain];
7. }

Experienced Cocoa developer’s have already discovered our error, please don’t ruin it for the rest of the readers. So after changing the code we run the test and…we fail. So from here Jake and I figured it out. Sleep won’t work because it stops the entire application, including the run loop that calls our timer, so we’ll just have to get the info out of the timer another way. We’re interested in testing that we’ve setup the timer correctly, and can trust that the timer just works. After a loooong time browsing documentation we figured it out. We could call this:

1. -(void) testTimerCallsGameAdvance {
2. MockGame *mockGame = [[MockGame alloc] init];
3. runner.game = mockGame;
5. [runner start];
6. [runner.timer fire];
8. STAssertTrue([mockGame advanceGenerationCalled], nil);
9. }

This will fire the timer and validate that it is setup correctly. So we did this, our test passed, we ran the app and it….failed. We knew the timer was executing because our test was passing, but when we ran the real app it just didn’t do it. Stumped and frustrated we stopped for the day. Then time passed…..

Yesterday I had a candidate in for an apprenticeship position, and we attempted to solve the same problem. We re-evaluated the test and decided calling fire directly was cheating. We looked into a half-dozen other ways to test this and learned about the NSRunLoop. Too much coding in Windows had rotted my brain, the run loop in Objective-C is not the same as the one in Windows. Specifically the current run loop is accessible as an object. We wrote a new test:

1. -(void) testTimerCallsGameAdvance {
2. MockGame *mockGame = [[MockGame alloc] init];
3. runner.game = mockGame;
5. [runner start];
6. [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow: 0.30]];
8. STAssertTrue([mockGame advanceGenerationCalled], nil);
9. }

It’s not perfect - the 0.30 relies on the hard coded default interval of 0.25, and that is too long, but it failed! In the process of learning about NSRunLoop we also realized I used the wrong method. The working code is this:

1. -(void) start {
2. timer = [[NSTimer scheduledTimerWithTimeInterval: interval
3. target:game
4. selector:@selector(advanceGeneration)
5. userInfo:nil
6. repeats:true] retain];
7. }

See the word “scheduled” in front of Timer? That actually adds the timer to the default run loop. Now we ran our spec, ran our actual code, and it works! The moral of the story? Well if you’re persistent enough you can get your code under test, and if your test is correct then your code will work.

Apprentice Architecture

by: paul | May 6th, 2009 | 1 comments »

Good design sense is a skill that comes only from many years of coding experience. Sometimes it can be very difficult to make those design decisions when you are an apprentice or otherwise don’t have much experience.

When I was an apprentice, oftentimes I would get a problem from Micah (my mentor) and would solve it the simplest way possible. Then I would walk up to him with my head held high and say I was done. He would almost always point out some way the architecture was not easy to extend or that he didn’t even know java had goto statements.

As an apprentice, the most important thing is to get everything to work correctly; only then should you concentrate on making sure it is good code. Writing great code with great designs is not a skill that will appear overnight. It took years of honing my design sense to be able to see designs right away, and I have only scratched the surface of my mentor’s abilities.

When I was an apprentice, I lacked the experience and skill to make difficult architectural choices. Let alone the ability to get it right on the first pass. I instead stayed close to the code and continued to move forward writing working code. This led to many design mistakes, and I had to throw out plenty of code or refactor to a better design. This is an exercise in taking a step backwards to move forwards. I learned the value of working code as well as the value of refactoring to better designs. Content before form is an important lesson I learned as an apprentice. It laid the foundation for the later lesson that good form is essential for high quality code.

As an apprentice, ask your mentor when you have completed a task if they think there is a better design. Then refactor that design and compare the two. Once again, repetition and practice will pay off as these exercises help you develop a matured design sense. You will learn from having to solve the problem without being told the design first, and then later having to re-implement a better design. This practice will internalize your design sense and eventually you will start to see the better designs the first time through.

The big problem with trying to get the design right the first time is it leads to coders block. Staring at the blank screen, trying to figure out the best way to do something. Even if you are writing the wrong code, it is important to be moving in some direction. When I see the code, oftentimes the design I get will form around the reading of the code rather than trying to think in the abstract about the whole problem at once. For me the problem and the design for the solution would be too big of a mental model to fit in my head all at once. Don’t worry, as you get more experience, your mental context will expand too. Your ability to sustain concentration will build as well as you practice and become more experienced. With experience, you won’t have to think about as many of the small problems anymore.

Architecture is like most craftsman practices in the sense that there are breakthroughs but there is always more to learn. I read a blog recently where Uncle Bob Martin talks about thinking through the architecture of a problem for hours or days before he will start. When I was an apprentice, I would not have had enough experience or a large enough mental model to think about the architectural impact of a solution. Even as a journeyman, I still only have a big enough mental model to think about an aspect of a problem, write the code, then reevaluate where the solution stands in the context of the system.

For an apprentice, bad design should not be seen as the end of the world. As long as you are learning and progressing in the techniques of creating working code, you are moving in the right direction. Design sense is a long-term skill and you will have to be patient with it. Remember that practice makes perfect.

Bug free or Free Bugs

by: paul | April 26th, 2009 | 1 comments »

I will not charge a client for a bug fix. Not a penny. If I make a mistake, it is my professional obligation to fix it.

If something doesn’t work with my car, there could be catastrophic consequences, I would be mad unless the company has a solution and offers to fix it for free. As a car owner, I expect nothing less. If a doctor makes a mistake, the patient has the right to sue the doctor for malpractice. Why is it if I make a mistake, I get off scot free? Or worse, I get paid to fix my own mistakes.

I know sometimes the devil is in the details. What is a bug? This is not an easy question to answer, but if there are customer written acceptance tests for the system, the items that fall through are fewer. These are general rules I go by when deciding if something is a bug or not.

One: I made a mistake. These are usually easy to notice, due to the redness in my face when the bug is reported. It is clear to me, I made a mistake, and it is clear I need to fix the bug. It should not cost the customer any money and the process should be transparent.

Two: There is a mistake that both development and the customer team should have caught. A scenario the customer should have specified and during development I should have noticed and brought to their attention. The fault goes on both parties and can be fixed at some version of half price (i.e. every other or a half price story).

Three: The application doesn’t behave correctly due to a missed specification. a scenario that the customer missed or a piece of the business logic that wasn’t fully correct. Almost everyone makes some mistakes. This is not a bug to me, but a feature enhancement. It can be written up as a story and completed.

I think it is important to build faith with the customer that there is a team of developers that are accountable. When you create the culture of accountability, it spreads. The customer team is willing to be accountable when they make a mistake. When no one is afraid to make or admit to a mistake, the projects quality is positively affected. Finally, by taking financial ownership of a bug, I build a trust with my stakeholders of the project.

Dependency Inversion Principle and iPhone

by: eric | April 16th, 2009 | 0 comments »

While working on the slides for our upcoming* talk on TDD for iPhone I asked Eric Meyer why we need the Dependency Inversion Principle. He eagerly answered, “Testing!” which made me laugh because I wasn’t giving him a trivia quiz, and I love Eric’s enthusiasm. He’s right, but the answer is circular. TDD is great because it gives us better designs that correspond to the DIP which enables TDD. Ha! I win, would you like to buy my consulting services? I have plenty of three letter acronyms where those came from.

Fortunately this isn’t the case, as the DIP existed well before TDD was popularized, but recently there was a bit of a dust-up over the SOLID principles. Many developers don’t understand why dependency inversion is important, so let’s demonstrate the usefulness in the Objective-C language on the iPhone.

The Dependency Inversion Principle (DIP) reads as follows:

A. High level modules should not depend upon low level modules. Both should depend upon abstractions.
B. Abstractions should not depend upon details. Details should depend upon abstractions.

So your code should depend on interfaces not on concrete classes, but why? Let’s look at a useful example in the Objective-C built in classes.

1. [NSString stringWithFormat:@"Object is: %@", object];

Do you see the DIP at work? It’s subtle here, but look at the method stringWithFormat, which takes a format string and then a list of parameters to fill in the format specifiers. The format specifier here is “%@” which takes any Objective-C object and calls descriptionWithLocale on it, or description otherwise. The object that is passed in here could be anything because all the stringWithFormat method cares about is that the object provides at least one of the two methods above. It’s depending on the interface, and not the concrete class. Let’s imagine a world where this didn’t exist, and the NSString method could only be composed of other strings. Suddenly you’d have something like:

1. return [NSString stringWithFormat:@"Object is %s", [object toString]];

That’s not bad at all is it? Well what if there are different objects.

1. return [NSString stringWithFormat:@"Object is %s %s %s", [object1 toString],
2. [object2 toS],
3. [object3 stringify]];

Are you starting to see the problem? Suddenly I can’t use this method without being cognizant of the specifics of the object I passed in. The minute one of these interfaces changes I have to change my code. I stole this method from some real code, so let’s look at it with some context. Once upon a time I was writing a mock object for an iPhone application. Now iPhone doesn’t have a good mock object framework, at least that I’m aware of, so I was rolling my own mock. The mock logged calls to it which I then queried. To construct the string I used this method:

1. +(NSString *) createStringFrom: (id) cell at: (CGPoint) point
2. sizeOf: (CGRect) rect
3. {
4. return [NSString stringWithFormat:@"%@ %f %f %f %f", cell,
5. point.x,
6. point.y,
7. rect.size.width,
8. rect.size.height];
9. }

Don’t sweat the details right now, they’re not important. Instead look at how you can see the DIP succeeding and failing in the same code here. The first parameter to createStringFrom is a cell object, but it’s of type id, so it could really be anything. That’s good because I wasn’t sure when I wrote this mock what type the cell would be and I’d generally prefer that my mock object not have to know the details. In fact the cell did change type several times which is why insulating yourself from change is so important. There’s an assumption by many developers that you should only worry about the things you’ll be changing in the future, but the best gauge of future changes is present changes, and if you’re changing a module frequently during development as you get it right you’ve found a likely change point. Even if you haven’t, you’ll make your life simpler in the present by insulating yourself from that change, speeding up your development cycles. I digress.

The rest of this method isn’t so lucky. You see CGPoint and CGRect are C structures, and I am left to write these as floats in the string (%f). If I was to change the parameter types frequently I would probably introduce some way to insulate myself from that change as well, possibly by wrapping them in objects. Since those parameters have not changed I haven’t done that, although I’m thinking about it.

Our current apprentice Colin has given a first-hand account of refactoring to DIP here as well.

How do I do it?

Eric and I are working on a program to run Conway’s Game of Life. On the iPhone screen you see Rounded Rect Buttons that change color based on whether a cell is alive or dead. The problem is that we don’t really want to drag and drop 300+ tiny buttons on the screen with interface builder. Instead we want to create them programatically, when the GameOfLifeViewController is loaded. Again I don’t want to overload you with extraneous details, so just take my word for it that there is a GameOfLifeViewController, and it needs to generate buttons. Here’s what my first pass at a test looked like in rough pseudocode:

1. for (int row = 0; row < 15; row++)
2. {
3. for (int column = 0; column < 15; column++)
4. {
5. button = GetButtonViewAt(row, column)
6. STAssertEquals(expectedSize, button.size)
7. STAssertEquals(expectedLocation, button.location)
8. STAssertAction(expectedAction, button.action)
9. }
10. }

There’s a lot wrong with this test. The most obvious is the hard coded 15. What happens when we want to change the number of rows and columns? Now we have to change code in multiple places. Naturally we did in fact change this, so I would have paid for this decision quickly if I hadn’t already refactored it away. The second is that there’s going to be more than one test for creation, each of which will have to perform the same 15x15 loop. The annoyance of constantly changing this will reveal the root the problem, the GameOfLifeViewController should really only be in charge of GameOfLifeView and really doesn’t need to know the details of the creation of its subviews. It’s violating the Single Responsibility Principle. We can fix that with a factory object, and will do so, but that will violate the Dependency Inversion Principle. So we’ve got one DIP violation, an SRP violation, and a potential second DIP violation, all demonstrated in 6 lines of test code.

Fortunately we’re going to introduce some interfaces to clean that up. The first is an interface to the board object. Why the board? Well the board should know the number of rows and columns it has, and if we can inject a fake version of the board we can actually set those to be 1 and 1. That’s will make our test focus on what we are truly concerned about. The second thing we’re going to do is introduce the Abstract Factory pattern to create our button views. It’s a bit of a heavyweight solution but its going to pay off by cleaning up our client code dramatically. We’ll have a View Controller concerned with controlling the view, and a creation factory concerned with creation, as well as a board that keeps track of rows. Let’s take a look at the test now, again in rough pseudocode.

1. -(void) setUp
2. {
3. controller = [[GameOfLifeViewController alloc] init];
4. controller.board = [[MockBoard alloc] init];
5. controller.buttonControllerFactory = [[MockButtonFactory alloc] init];
6. }
8. -(void) testCreationOfButtons
9. {
10. [controller viewDidLoad];
11. bool called = [controller.board calledWith: params that: I want: true];
13. STAssertTrue(called);
14. }

So what’s going on here? In the setup we load up the board with a fake board, one that returns 1 for its rows and columns, and we load up its buttonControllerFactory with a fake buttonController Factory. In our test we call the viewDidLoad method, which is called on controllers after their view is loaded. Duh. That’s the method where we use the buttonControllerFactory. Then we call a method on the mock factory calledWith: that takes the parameters I expected the creation factory to receive. Finally we check that it was called with the assertion. So far I’ve only been showing test code, because I wanted to demonstrate just how much cleaner our tests have become. Now there’s no loop, no hard coded constants, and no testing the internals of the creation factory in the view controller. That isn’t to say we don’t test the creation, we just do that in a different test case, the one that corresponds to the ConcreteButtonFactory class. As a rule if our test code is cleaner, our implemenation code is cleaner, and that’s the case here as well.

Let’s now look at the real class, and how to enable Dependency Inversion on it. We have to start with Objective-C protocols. Protocols are very similar to Java interfaces, and just as useful, but are strangely underrepresented in XCode. As best I can tell** just create an NSObject subclass and remove the .m file and any imports from the .h. Here’s the protocol from the unfortunately named ButtonControllerFactoryProtocol.h:

1. #import "ButtonController.h"
3. @protocol ButtonControllerFactoryProtocol
5. -(ButtonController*) createButtonControllerForCell: (id) cell
6. at: (CGPoint) point
7. sizeOf: (CGRect) rect;
9. @end

As you can see the interface only defines one message, createButtonControllerForCell. It returns the ButtonController object, although it could return an object conforming to a protocol. Indeed one could argue it should, as that would streamline my mock object. Hmmm… I see a refactoring in my future. Now let’s look at the class definition for the ConcreteButtonControllerFactory, which is the object that is actually used when the real app is running.

1. #import <uikit />
2. #import "ButtonController.h"
3. #import "ButtonControllerFactoryProtocol.h"
5. @interface ConcreteButtonControllerFactory :
6. NSObject<buttoncontrollerfactoryprotocol> {
8. }
10. @end

Not much here is there. The important part to look at is NSObject. ConcreteButtonControllerFactory inherits from NSObject, as all Cocoa objects do, and conforms to the ButtonControllerFactoryProtocol. Like a Java interface this is a contract to later implement createButtonControllerForCell in my implementation file and that’s exactly what I’ve done.

1. #import "ConcreteButtonControllerFactory.h"
3. @implementation ConcreteButtonControllerFactory
5. -(ButtonController*) createButtonControllerForCell: (id) cell
6. at: (CGPoint) point
7. sizeOf: (CGRect) rect;
8. {
9. ButtonController *controller =
10. [[ButtonController alloc] initWithCell:cell];
12. UIButton *button = [UIButton buttonWithType:UIButtonTypeRoundedRect];
13. [button setFrame:rect];
14. [button setCenter:point];
15. [button setImage:[UIImage imageNamed:@"dead_cell.png"]
16. forState:UIControlStateNormal];
18. [button addTarget:controller
19. action:@selector(bringToLife:)
20. forControlEvents:UIControlEventTouchUpInside];
22. controller.view = button;
23. return controller;
24. }
26. @end

This isn’t an article on UIKit so I’m not going to get into the details of creating the button. You can see again that I’ve ignored the DIP for creating the UIButton and the ButtonController. Since this is a factory I’m okay with this violation. I could have created abstract factories for the UIButton and ButtonController and passed them in here but I don’t believe I’d gain any benefit from that implementation. That’s verified by my testing, as writing tests for the creation of these objects was hard before I introduced this factory but isn’t any longer. If it becomes difficult then I’ll consider making the change to two factories. Now we need to inject the concrete dependency. In a Java project you could do this in a main method or with a DI framework like Spring. We’re going to be using my favorite DI framework of all time: Interface Builder!*

It’s a common misconception that IB is just a GUI builder. What’s great about it is that it doesn’t just generate a bunch of code, but instead “freeze dries” real objects to be instantiated when the program loads the NIB file. Furthermore while the most common Outlets are UI elements there’s no reason other objects can’t be used. Let’s take a look at my NIB document window for the GameOfLifeViewController:

As you can see I’ve got objects for the Game, ConcreteButtonControllerFactory, and the Board. That’s because all three of the objects are injected at runtime, while the code depends on protocols. To add your own objects to a NIB file simply by selecting it from the Library:

It’s inspector window will have a class drop down, so set this object’s object to your class. Finally you need to connect the outlets to actual outlets in your code. Let’s look at the Outlets tab in IB for my controller (the File’s Owner):

Notice how the controller still thinks of them all as NSObject? So if I change the objects it doesn’t care, as long as I conform to the interface? Finally let’s take a peek at the controller interface file:

1. #import "BoardProtocol.h"
2. #import "ButtonControllerFactoryProtocol.h"
3. #import "GameProtocol.h"
5. @interface GameOfLifeViewController : UIViewController {
6. IBOutlet NSObject<boardprotocol> *board;
7. IBOutlet NSObject<buttoncontrollerfactoryprotocol> *buttonFactory;
8. IBOutlet NSObject<gameprotocol> *game;
9. }
11. @property(nonatomic, retain) NSObject<boardprotocol> *board;
12. @property(nonatomic, retain) NSObject<buttoncontrollerfactoryprotocol> *buttonFactory;
13. @property(nonatomic, retain) NSObject<gameprotocol> *game;

See any dependencies on the actual objects? Nope - just protocols. There are properties for the objects so that I can inject the dependencies through the tests, all of which pass in fake versions as various times depending on how it suits the test. It may be unhealthy - but I find this incredibly cool.

Summary

What I love about Interface Builder is setting this up is extremely simple. Writing tests for the controller that is based on these dependencies is also extremely simple, far simpler than just depending on the concrete objects. It’s funny how developers often complain about the time wasted writing tests or “over-engineering” in this way, but once I broke these dependencies I got things done a lot faster. I’m writing working code without loading the actual simulator. I write my tests first, making progress, and when I’m done for the session I run the app once or twice just to verify I haven’t done anything like forgotten to inject the dependency. Compare that to running the simulator constantly, each time hoping your code is functional. Who’s wasting their time now?

FootNotes:
* Uh yeah we gave that talk already. These take a while!
** I’m 99% positive there’s a better way, and that I haven’t found it. Tell me if there is!
*** To be fair it’s the only one I’ve actually used.

Copyrights Conundrum

by: micah | March 12th, 2009 | 0 comments »

Here’s the situation. You’ve spent days working on an open source project. Your sweat, blood, and tears have been poured into thousands of lines of code spread through countless file in a sprawling tree of directories. The final step is to release your masterpiece into the wild. So you choose your open source license and then it hits you; you realize that need to add a copyright header to EACH and EVERY source code files. How tedious! You google for a tool that will automatically add the headers for you, and you find nothing satisfactory. In the end, you write your own script to add the headers for you.

I’ve been in this exact situation and written such a script too many times. My most recent experience and script implementation will be the last. I’ve dubbed this last imeplementation MM Copyrights and you’re welcome to use it.

MM Copyrights is a simple Ruby gem that will search a directory for source code files, inserting or updating a header comment in each file.

Installing

It’s just a matter of installing the gem hosted on Github.

1. gem sources -a http://gems.github.com
2. sudo gem install slagyr-mmcopyrights

Update: The github shenanigans is not longer needed. To install, simply:

1. sudo gem install mmcopyrights

Usage

Let’s say you want to add the following copyright header to all .rb files in the lib directory…

1. require 'mmcopyrights'
2. MM::Copyrights.process("lib", "rb", "#-", "©2009 Micah Martin\nAll rights reserved")

And all the ruby files will look like this:

1. #- ©2009 Micah Martin
2. #- All rights reserved
4. ... ruby code ...

Typically I keep the copyright text in a separate file and write the following Rake task…

1. task :copyrights do
2. require 'mmcopyrights'
3. MM::Copyrights.process('lib', "rb", "#-", IO.read('copyrights.txt'))
4. end

or the following ant task if need be…

1. <target name="copyrights">
2. <exec command="ruby">
3. <arg value="-rrubygems" />
4. <arg value="-e" />
5. <arg value="require 'mmcopyrights'; MM::Copyrights.process('src', 'java', '//-', IO.read('copyright.txt'))" />
6. </exec>
7. </target>

Details

If you haven’t guessed, MM::Copyrights::process takes 4 parameters.

1. The path of the directory containing source code.
2. The extension of source file to be processed.
3. The comment prefix. Note that there’s an extra -. This is because we want the copyright comments to be unique from all other comments. The tool will search for this unique prefix to find existing copyright headers.
4. The copyright text, without the comment prefix.

That’s about it. Feel free to run MM::Copyrights::process multiple times on the same code base. It’s harmless. And when the new year rolls around, just change the copyright text and rerun the tool. It will update existing headers.

When is a Mock not a Mock

by: eric | January 10th, 2009 | 0 comments »

A couple of weeks ago I had to add a feature to some code that was written almost two years ago. It was a pretty simple web service client that constructed some XML, using Ruby’s REXML library, and made the web service call via another library. A quick inventory shows two dependencies, the web service call and REXML, so that’s what I expected to see when I came across the tests. But what did I see? Paraphrasing:

1. @client_under_test = Client.new
2. @client_under_test.stub!(:send_message)
3. @client_under_test.stub!(:parse_response)

Of course the send_message call contained the construction of the XML so that there was no test that the XML being constructed was correct. Heck that was all that was being done in the class, and a closer inspection of the tests revealed that all they were doing was testing the error handling by changing the return value of parse_response. This is a classic sign that the developer at the time couldn’t figure out how to stub out the actual dependency, and so they did the next best thing, they stubbed out the caller.

I knew this was wrong, but I had an iteration meeting the next day and still had to update the .NET web service that this code was calling. So I started to just go ahead and make the untested change, but then my Clean Code wristband started burning until I couldn’t take it any more. I had to get the REXML library under test. So what to do? Well the REXML construction looked like this:

1. doc = REXML::Document.new(create_schema)
2. doc.elements["name1"].text = "thing 1"
3. doc.elements["name2"].text = "thing 2"
4. ...

Okay I know how to do this normally. The first test was simple:

1. REXML::Document.should_receive(:new).with(schema)

It passed! What was so hard about testing this anyway? Okay let’s test that the elements are set:

1. ....
2. elements = mock(REXML::Document, :[] => mock_element)
3. doc = mock(REXML::Document, :elements => elements)
4. REXML::Document.stub!(:new).with(schema).and_return(doc)
5. ...

Oh that sucks! Since the API doesn’t run entirely through the doc object I have to stub elements, then stub each individual element, then should_receive on text, and if I want to be perfect I can stub out the array accessor too. I have to should_receive for each element name, the tests will be hard to follow and fragile. Now I get why the original developer punted on this. There’s got to be a better way. I can start expecting actual XML, but from past experience I know that leads to very hard to read tests that give errors that aren’t easy to diagnose. What I really care about is that the elements in this hash-like structure have their text element set to the right data. After trying out several approaches I had a moment of inspiration, I want a struct! Must be my inner C programmer. If elements was a hash of structs, I could just check their text values are set. This way I’m testing that I’m using the REXML library right rather than testing the library itself. If only Ruby had a struct concept…..

I suppose you’ve already figured out that it does, haven’t you?

1. MockElement = Struct.new("MockElement", :text)

What that does is setup a class named MockElement, with the accessor text. When is a Mock not a Mock? When it’s really a Fake. Instead of stubbing out elements to return yet another mock, elements will return a hash of the element names I know I’m using that point to these fake elements. Then after making the call in the client, I’ll just check that the text is set properly. This is best demonstrated with an example:

1. before(:each) do
2. @elements = initial_elements_hash
3. @doc = mock(REXML::Document, :elements => @elements)
4. REXML::Document.stub!(:new).with(schema).and_return(@doc)
5. end
7. def initial_elements_hash
8. return {"element1" => MockElement.new,
9. "element2" => MockElement.new}
10. end
12. it "should set the data in the xml doc" do
13. @client.send_message
15. @doc.elements["element1"].text.should == "Eric"
16. @doc.elements["element2"].text.should == "Smith"
17. end

Aha! Now I’ve tested that the XML was properly constructed without relying on looking at the actual XML text itself and without stubbing out the class itself. The code is cleaner, the tests are easier to understand, and I can implement my new feature. More importantly, my wristband stopped burning.

Embedded Ruby Talks Morse

by: doug | January 8th, 2009 | 7 comments »

I've been working with Ruby on embedded systems for a couple of weeks now. The final pieces have fallen into place and I'd like to share them with you. From the beginning of the project, my goal have been to turn some LEDs on and off from a web page.

Getting down to the metal

Any embedded software application has to eventually control some interesting hardware. My ARM board had two LEDs wired up, so I thought I'd try to control them. The control for the ports that control those LEDs are in memory mapped registers. Ruby really never let's us see the memory that we are working with, and even if it did, we are working in a virtual memory environment, so we wouldn't be able to access the specific memory address where the register is located. So it was time to learn how to write Ruby extensions.

I was surprised how easy this ended up being. Ruby has a great little module 'mkmf' that makes Makefiles for you. The API for defining ruby modules was also pretty straight forward and explained well in Pickaxe. I put the module I wrote up on git hub, so take a look there if you are interested. The extension allows you to just read from or write to any physical address on the device. To set a bit in a register, you might do something like this.

1. include MemoryMappedIO
2. from_register = read(0x560000010, "w")
3. write(0x560000010, "w", from_register | 0x00000001)

Ahh. . 0x, hex numbers, addresses. It's nice to be back.

To get the module into my Ruby environment, I created an 'mmio' directory in the 'ext' directory of the Ruby source and dropped in the source files. I didn't have to change any makefiles, I just created an extconf.rb and the ruby build system took care of the rest. It would certainly be possible to build the module separate from the rest of the Ruby source, but since I already had the source tree configure to cross-compile, the path of least resistance was to just include this extension in the source tree.

I posted my extension to the ruby-core mailing list to see if there was any wider interested in it. It turns out there is a memory mapper extension already written. I took a quick look and it isn't exactly the same thing that I did. My interface is much more basic (and simple), but I think the mmap extension could be made to do the same thing I am doing.

The Promised Land

With a way to control the hardware from Ruby, I had finally arrived where I had wanted to be. The entire thesis behind this little experiment was that Ruby could make embedded application development fast and fun. I found both to be true. This entire application came together in just a couple of hours. I was able to make use of a ruby gem that did a bunch of the work for me. I called the application weblink. You are free to interpret the name however you would like (We Blink, Web Blink, Web link). The source is on git hub.

When test-driving an embedded system, you'll save yourself a bundle of time by doing as much of the development as you can on your host computer. The download, restart, test cycle can really add up when you are working in quick TDD cycles. On my particular system, the files system is still being hosted remotely, and the language is interpreted (no compile) so this overhead is not that bad. Still I wanted my application to run on my development machine, and I didn't think it wise to overwrite some random address in my Mac's memory, so I needed an abstraction.

When developing these kinds of environments in a C++ system, I would traditionally build a pure-virtual interface and implement it twice, once for the target hardware and once for the simulated, or development environment. In C you can do a similar thing with the linker. You write two functions, one for each platform and just link in the correct one when building for that platform. With Ruby's ducktyping, there is no need for a defined interface, but the concept is very similar.

I wrote two Led classes both with 'on', 'off', and 'on?' methods. The MockLed class just saves the on/off state in a YAML file. The real Led class uses the mmio extension to read and write a bit in a memory mapped register. I did not want the real Led class to have the platform specific knowledge about which LEDs were in which register, so I used the Factory Pattern to create the Led objects. The factory method is called 'find.' The MockLed file implements this same method that returns MockLed objects in the development environment.

The tricky part of this is switching between the two Led classes. For this, I chose to use Sinatra's environment configuration scheme. The production environment is used when running the app on the target, and the development environment runs on my Mac.

1. configure :production do
2. require 'leds_S3C2440A'
3. end
4. configure :development do
5. require 'mock_led'
6. end

Dit, Dash

To celebrate the 170th anniversary of the first demonstration of the telegraph, I decided to turn the application into a visual telegraph, beeping out little messages. The ideas is that you enter a message in the web page, and the LED blinks out your morse code. Here is where I saw the great power of not only Ruby, but also the community who uses it. I discovered right away that there is a Morse Code Gem already written. You give it a string and It gives you back a series of dits, dashes, and spaces. Thanks Ben!

Eric Meyer sat down with me and we ping-ponged out the morse blinker in short order. I love how simple this class ended up. It just proves again to me what great code you can write in Ruby.

See it for yourself

Okay, so enough talking, let me show it to you. Enjoy.

A Screenshot of the app:

What's Next

So I know that the response that I will get from embedded developers is "Great, but it's slow and big, it can't be used in a real system." I'm going to do some profiling and optimization on this system to get some hard performance data. Initial results (just looking at 'top') shows that the ruby process with Sinatra, Webrick, the morse gem, and my code loaded up is taking up about 12% of the system memory. That mean about 6 or 7 MB. What nice is that you only pay for what you use. If you don't require it, it won't load into memory.

As far as clock cycles go, the ruby processes peaks out at about 20% of the 400 MHz processor when serving HTTP requests. While blinking the LEDs, only a fraction of a percent of the processor's cycles are being consumed. I'm not happy with the performance of Webrick (no one ever claimed it was fast anyway) and I am going to try to get Sinatra running with 'Thin' instead.

Synchronization: I fork a new ruby process to blink out the messages. If you got both LEDs blinking at the same time, they could step on each other's toes and cause incorrect values to be written to the LED. I need to stick a mutex in to protect that register.

I remember early in my career, Kevin Moore and I had started using Ruby to run our embedded builds. We used the Win32 API to drive the automation interface of Metrowerks and we wrote lots of little tools to do the annoying parts of building a flash image for us. I remember at the time, Kevin and I having a conversation that went something like "Wouldn't it be cool if we could write the entire app in Ruby?" Well Kevin, now we can.

Embedded Ruby Sings Sinatra

by: doug | December 30th, 2008 | 3 comments »

Last Time I got ruby up and running on my Arm based embedded development platform. Here is a quick summary of what Santa and I did over Christmas.

Sockets

Getting support for sockets built into Ruby turned out to not be hard at all. All I had to do has uncomment the extensions I wanted in ext/Setup. Here is everything I decided to turn on.

1. etc
2. fcntl
3. iconv
4. socket
5. stringio
6. strscan
7. syck
8. thread
9. zlib

After a rebuild and reinstall, I could require and use 'socket.' Welcome, Ruby, to the outside world!

Relocating Ruby

I decided to move ruby to the 'standard' install directory (/usr/local/) instead of the root that I had done in the first iteration. This would avoid having to set the RUBYLIB environment variable and keep any mess I made out out of the main system /bin and /lib directories. First I removed the --prefix from the configure script I had written last time. This will cause Ruby to build assuming the default install location (/usr/local/).

1. #! /bin/sh
2. export ARM_TOOLS=/usr/local/arm/gcc-4.2.3-glibc-2.3.3/arm-unknown-linux-gnu/bin
3. export CC=$ARM_TOOLS/arm-unknown-linux-gnu-gcc
4. export LD=$ARM_TOOLS/arm-unknown-linux-gnu-gcc
5. export AR=$ARM_TOOLS/arm-unknown-linux-gnu-ar
6. export RANLIB=$ARM_TOOLS/arm-unknown-linux-gnu-ranlib
7. export ac_cv_func_getpgrp_void=yes
8. export ac_cv_func_setpgrp_void=yes
9. ./configure --host=arm-unknown-linux --enable-wide-getaddrinfo

After Ruby was built, but before installing it (make install), I hacked the DESTDIR in rbconfig.rb

1. ...
2. TOPDIR = File.dirname(__FILE__).chomp!("/lib/ruby/1.8/arm-linux")
3. DESTDIR = "/~/ruby/install"
4. CONFIG = {}
5. ...

This created the /usr/local tree inside my install directory. I created a /usr/local/ on my file system and copied that tree over. The only problem with this process is that the sh-bang line on the top of all of the ruby scrips (irb, testrb, etc.) had the wrong path in them. They had the install path on the host machine (/~/ruby/install/usr/local/bin/ruby) instead of the actual path on the target (/usr/local/bin/ruby). I changed those by hand.

Sinatra

Ahh, now the good part. Using a gem on an embedded system is an interesting problem. I first had to get rubygems installed on the network files system. I downloaded the source code and temporarily placed in on the target files system. On the target I ran the install command.

1. ruby setup.rb --no-rdoc --no-ri

I found it more convenient to install the gems from the host system instead of the target. It would be faster and the 'gem' command had some ruby dependencies that I hadn't yet build. I downloaded Sinatra and Rack and then manually installed them into my target's file system on the host machine. Note that /arm/fs/ is the root of my networked file system. I switched off rdoc and ri to help keep the install small.

1. sudo gem install sinatra-0.3.2.gem -i /arm/fs/usr/local/lib/ruby/gems/1.8 --no-rdoc --no-ri
2. sudo gem install rack-0.4.0.gem -i /arm/fs/usr/local/lib/ruby/gems/1.8 --no-rdoc --no-ri

With Sinatra installed and ready to go, I wrote a quick Sinatra app. Sinatra runs on Mongrel by default, but I wanted to run on the lighter weight Webrick web server which was already in my ruby installation.

1. require 'rubygems'
2. require 'rack/handler/webrick'
3. require 'sinatra'
4. Sinatra::Application.default_options.merge!(
5. :run => false,
6. :env => :production,
7. :port => 80
8. )
9. get '/' do
10. "Hello Sinatra!"
11. end
12. Rack::Handler::WEBrick.run Sinatra.application

Drum roll please ...

Next Steps

I've been plugging away on the linux device and ruby extension side of the project. I'm still on the steep side of the learning curve, but making progress. I'm still after that elusive blinking LED!

Embedded Ruby - First Steps

by: doug | December 19th, 2008 | 3 comments »

I spent the beginning of my career working on embedded systems for some audio company. These embedded systems were my first love in programming and to me, no flashy css on a web page can beat the thrill of writing some code that flashes an LED. I've been working lately with Ruby a lot. I love it. I love that the language nearly disappears and I am able to express pure intent. Ok, so there is still syntax and such, but I can create so much more with so much less code.
I hate living a fragmented existence, so I've been working the last few days on bridging these two worlds of mine. I figured that it will also give me a chance to get more familiar with linux and the latest in the embedded world.

First up, pick a platform.

I chose this single board computer from Glomation. It has an ARM920T core and an LCD / touch screen interface which may be fun later.

Next, Tools.

I was determined that I could get a toolchain up and running on my MAC, but eventually ended up running linux in a virtual machine. Glomation had a set of tools and a pre-patched kernel already to go on their support page and our apprentice had a virtualbox linux image that he let me copy, so I was up and running quickly.
I do want to share with you a few of the resources I found in the processes.

• I began with the GNUARM project.
• Tom Walsh on the gnuarm mailing list was helpful and pointed me to his scripts. I came real close to a working toolchain on the Mac with his help.
• I also gave the Crosstools project a try but ended up in the same place I was before.
• Thanks to David Goodlad's RubyConf 2008 presentation I discovered Open Embedded. This seems to be where the momentum is right now. I imagine I will give this project a careful look soon. I wish I would have started here.

Cross-Compiling Ruby

A few folks have had some success cross compiling ruby and Goodlad's presentation claimed that ruby is a part of the open embedded project. I built ruby 1.8.6-p111 because that what was on my linux distribution and it seems to be a well distributed release. Here is the script I used to configure ruby for cross compilation. Thanks to The Zen Machine blog for the skeleton of this script.

1. #! /bin/sh
2. export ARM_TOOLS=/usr/.../arm-unknown-linux-gnu/bin
3. export CC=$ARM_TOOLS/arm-unknown-linux-gnu-gcc
4. export LD=$ARM_TOOLS/arm-unknown-linux-gnu-gcc
5. export AR=$ARM_TOOLS/arm-unknown-linux-gnu-ar
6. export RANLIB=$ARM_TOOLS/arm-unknown-linux-gnu-ranlib
7. export ac_cv_func_getpgrp_void=yes
8. export ac_cv_func_setpgrp_void=yes
9. ./configure --host=arm-unknown-linux --prefix=$HOME/ruby/install

ARM_TOOLS is where I installed the cross compiler. The prefix is just the place where 'make install' will drop the results of the build. The configure script generates a file in the ruby root director called fake.rb that causes some problems when I went to 'make'

1. /ruby-1.8.6-p111/fake.rb:12: unterminated string meets end of file (SyntaxError)

I opened it up and it has a very obvious syntax error in it. I added the missing backslash on line 12 and ruby built fine from there.

1. 12 ALT_SEPARATOR = "\";

becomes . . .

1. 12 ALT_SEPARATOR = "\\";

'make' then 'make install' and I had my cross compiled ruby

Getting Ruby onto the Target

The last step to seeing little rubies flying on my ARM core was to get the build onto the target. The Glomation board came preloaded with a kernel and a root file system. The kernel source was provided on the support page but not the root file system. I asked their support guy and he kindly posted it for me. I was dreading having to rebuild a file system from scratch, so I was grateful for his help.
I decided that the best option would be to just have the target load of the root file system over the network on boot. I followed these instructions for mounting the image on my linux VM and these instructions for setting up an NFS Server. Mounting the file system over the network is a great option because now I can edit the file system from my Mac or linux VM and have the target see the changes without even rebooting. Perfect! The 2440 board came loaded with UBoot and all I had to do to load the file system over the network was to set the nfsroot in the bootargs from the bootloader's command line.

1. set bootargs 'nfsroot=10.0.1.151:/arm/fs rw console=ttySAC0,115200 ip=dhcp init=/linuxrc'

Once booted, I only needed to set the RUBYLIB environment variable so that ruby knows where to find the libraries. I added this to my init script.

1. export RUBYLIB=/lib/ruby/1.8

And there you have it. Ruby running on my ARM board:

Next Steps

Here is where I want to go from here.

• Sockets. I need to build openssl support for ruby so that I can talk to the world from Ruby. The Zen Machine ought to help me out here.
• Sinatra - I'd like to run Sinatra on the target and start serving up some web pages
• RTC - I want to get that on chip Real Time Clock working so that it's not always 1970
• Drivers and extensions - I want to be able to control some hardware from Ruby. I've got a lot to learn here but I'll have to find or write a linux driver and then build a Ruby extension on top of it. I'm imagining something like led.on and led.flash.
• Submit to ESC. I'm hoping to submit my work to the Embedded Systems Conference. I hope that Ruby can start picking up some momentum in the space and save embedded developers gobs of time like it has done in the web development world.

Thanks for reading. Let me know if you have any ideas or suggestions!

One Take on Configuring Rails Routes and asset_host

by: jim | December 2nd, 2008 | 1 comments »

We recently had an interesting requirement surface. In anticipation of the release of a number of demo environments, our customer requested that system configuration be able to be done at the server level. The goal was to avoid being forced to use source control to manage configuration files.

A little background is in order. The systems that we are working on are pretty large. There are two distinct, decoupled systems, each consisting of between 10 and 40 subsystems. Of these subsystems, a handful on each side are Rails applications. Most others are small Ruby application that talk a central Rinda server. We already have a configuration strategy in place for the two systems, which involves a central configuration file per system which populates a globally-accessible configuration hash.

The directory structure looks something like this:

/deployment
/deployment/system_one
/deployment/system_one/etc
/deployment/system_one/etc/config.rb
/deployment/system_one/rails_app_one
/deployment/system_one/non_rails_subsystem_one

The systemone/etc/config.rb file creates the configuration hash and populates it with all of the appropriate configurations (in practice, config.rb actually loads the appropriate configuration file based on an environment variable, i.e. RAILSENV, but we will ignore that here for brevity and clarity):

1. APPLICATION_CONFIGURATION ||= {}
2. APPLICATION_CONFIGURATION[:foo] = "bar"

From the context of a Rails application, this central configuration file is loaded from railsappone/config/environment.rb:

1. # Load the system configuration
2. require File.expand_path("/deployment/system_one/etc/config")
4. # Bootstrap the Rails environment, frameworks, and default configuration
5. require File.join(File.dirname(__FILE__), 'boot')

With this configuration strategy in place already, the team was tasked with setting up multiple demo environments for the entire system, each deployed on a different server and accessed through a different external URL. We could have attempted to solve the configuration issues by just adding many separate Rails environments, say demo1 and demo2, but there were a couple problems with that. First, the configurations for the different environments would have been nearly identical. Second, we wanted to avoid the complexity of relying on source control to manage the configurations for each deployment. In most cases, relying on the Rails convention of setting up a new environment makes a lot of sense. However, in our case, the only differences between the environments would be setting the asset_host and routes. Storing this in source control means that in order to make a change to the external URL means that files under source control need to be modified, checked in and then redeployed to the affected system. It makes much more sense to have some reasonable defaults in source control and then provide a mechanism to override these configurations at the server level.

The solution to this problem ended up being quite simple. We first agreed on an acceptable location for the server configuration file:

deployment/config
deployment/config/server_config.rb

In order to allow for overriding the system configuration, this code was added to /deployment/system_one/etc/config.rb:

1. # Allow for configuration to be overridden by a config file that is not under source control
2. server_config_file = File.expand_path("/deployment/config/server_config.rb")
3. require server_config_file if File.exists?(server_config_file)

The server configuration in deployment/config/serverconfig.rb will be loaded (and executed), if it exists. If it doesn’t exist, the default configurations will be used, which is the desirable behavior. The first task was to configure the assethost. To start with, we add the desired assethost configuration to /deployment/config/serverconfig.rb

1. APPLICATION_CONFIGURATION[:my_rails_app_asset_host] = "https://www.example.com/my_rails_app/demo"

For each Rails environment that we want the assethost to be configurable from the server configuration file, just add this line to the appropriate config file, e.g. deployment/systemone/railsappone/config/environments/demo.rb:

1. config.action_controller.asset_host = APPLICATION_CONFIGURATION[:my_rails_app_asset_host]

You would want to add a separate assethost configuration for each Rails application, and each Rails application would set its own assethost to the appropriate configuration. For a simple configuration like asset_host, this works great. For routes, though, it gets a bit more complicated. We need a Mapper instance in order to build a route. For example, your routes configuration looks something like this:

1. ActionController::Routing::Routes.draw do |map|
2. map.connect ':controller/:action/:id'
3. end

We initially saw two options. 1) Create a data structure representing the desired routes, store it in the configuration, then from the routes file iterate through the structure created in the configuration file, creating the appropriate routes; or 2) Store some Ruby code in the configuration so that the context (i.e. the Mapper instance) can be passed to the block at the run-time. This second option eliminates the need for a secondary data structure to represent the routes. How better to configure the routes than with the actual code used to configure them?

It turned out that this was nearly as simple as the assethost configuration. First, let’s define the routes in /deployment/config/serverconfig.rb:

1. APPLICATION_CONFIGURATION[:my_rails_app_routes] = lambda do |map|
2. map.connect 'my_rails_app/demo/:controller/:action/:id'
3. map.connect 'my_rails_app/demo', :controller => "login", :action => "index"
4. map.login 'my_rails_app/demo/login', :controller => "login", :action => "index"
5. end

We have used the lambda method to convert a block into a Proc object. The Proc object is stored as a value in a hash which can be executed later. The rest should look familiar to anyone using Rails; it is exactly what we would have had in our routes file. Now, from /deployment/systemone/railsapp_one/config/routes.rb, we can do this:

1. ActionController::Routing::Routes.draw do |map|
2. APPLICATION_CONFIGURATION[:my_rails_app_routes].call(map) if APPLICATION_CONFIGURATION[:my_rails_app_routes]
3. # Install the default route as the lowest priority.
4. map.connect ':controller/:action/:id.:format'
5. end

In order to invoke the Proc object stored in serverconfig.rb, we just send the call message to the Proc stored in the configuration hash, if it exists. In our implementation, we invoke the Proc configured in serverconfig.rb before all of the other (default) routes, with the assumption that the configured routes should have the highest priority. If we run into a case where this isn’t the case, we can address that problem then.

###############