8th Light Blog

There is no perfect software. At least I have never seen it. Bugs and minor feature changes are indications people are using your software. Real users hit a system in ways that no control group can, and on non-critical applications, this is the best way to test your software. Let people use it and see what happens. This is goes in line with the agile philosophy of release early and often. Get your application out there as fast as you can, so you can mold the finishing touches around the real users experience rather than a faux-environment.

There is some conversation about what is and what is not a “bug” in the software world. That is not a conversation I would like to partake in here, so lets call both bugs, integration items, minor feature enhancements, and things that fall through the cracks of development tweaks. It doesn’t matter what the nature of origin is, these are all things that MUST get done.

After the release of one of our products, a load of tweaks came in from the customer. As proud craftsman, we decided tweaks were our responsibility, and we would take them on in addition to our normal iteration. So we started to do them, to the detriment of our iteration. We accomplished only about half of our iteration’s velocity.

The next iteration, much to our surprise, we were twice as busy with production support. This is about the time that a developer looses a little faith. What did we miss? Is this high quality software we are writing? So we lost even more velocity when it came to iteration 2 after the release. Also, the customers were now unable to accurately plan new features moving forward due to an unstable velocity.

It is so hard to predict or estimate production support and tweaks. However, we needed to be able to so that the production support didn’t leave such a footprint in the project. It felt and looked like we were not getting very much done, even though we were working harder than usual. It was the time being put into a vacuum and being unaccounted for that was troubling the project. It also had a negative effect on the morale of the team.

We came up with a card, we call the “Production Support Card.” The amount of the iteration’s velocity this card took up was calculated by the amount of time we spent on production support the previous iteration averaged with the amount of time allocated for that iteration(sound like a familiar formula?). It is added as a card to the next iteration. If the developers only spend 6 of the 10 points on production support, it is expected that they will complete 4 points worth of stories, which are automatically entered in the iteration. For the first iteration where it becomes apparent that we need a production support card, we set the point value of the card at 0 and track how much time we spend, bumping out of the iteration the least important stories if needed.

So, what does this tracking buy you, if you have to spend the same amount of time on tweaks? First, it allows transparency to the customer about what you are working on that week. When they see your normal velocity of 20 points turn into 5 points, they have a right to be worried. When you say, in a defeated voice “we were fixing bugs,” they also have a right to worry about the stability of the code you have been writing, even though this spike in minor changes to the application is a part of the normal process.

Second, it raises the moral of the team, because they are working towards a specific goal, to remove the production support cards from the iteration. Also, we get the satisfaction of maintaining a velocity in points, which is something we know so well it is hard to work without.

It takes a few iterations, and the team squeezes the life out of the production support card, putting you back on track. After those iterations, the footprint goes from sasquatch to mini-me.

It also helps the customer plan around production support. Their time lines and release dates are made from a projection of feature difficulty to development’s velocity. Over a long period of time, the velocity normalizes, and it hurts the projections to have hiccups. If you have production support data, you can predict about how much time around a release you will loose on the initial release of brand new development.

Background

When building a software system composed of multiple decoupled components, the need typically arises for interprocess coordination and communication. As an example, say that we have a customer requirement that allows emails to be sent based on certain events or conditions that occur while the user is interacting with a web application. We build an email component that takes care of the details of actually sending of the email. The web application could use the email component directly, but that would introduce a dependency that we’d rather avoid. How can the two processes communicate without creating a dependency between the two?

If you’re using Ruby, there’s an option that is built right into the standard library. The Rinda module implements the Linda distributed computing paradigm in Ruby. That’s great, but what is Linda? Linda is a coordination language developed by a couple of Yale researchers in the early 1990s. Linda itself is built on the concept of a tuplespace. A tuple is an ordered list of objects, and a tuplespace contains a set of tuples which can be accessed concurrently. A basic set of operations were defined by the language to allow reads, writes and takes.

Getting back to the example, how can Rinda be utilized to enable communication between the web application and the email component? First, a tuplespace will need to be setup that both components can access. When a condition arises within the web application that requires an email to be sent on its behalf, a specific tuple will be written to the tuplespace. The email component is looking for tuples that match a specific pattern. When it finds the tuple written by the web application that matches this pattern, it processes the request and reports back to the tuplespace when finished. In this way, we end up with only data coupling between the two processes, which is a much looser coupling than a direct dependency. This type of approach bears some resemblance to a blackboard system.

Now that we know a little bit about what Rinda can buy us, let’s dive in and try it out.

Hello World

Let’s put together a simple server and client to demonstrate how each works.

Here’s a simple server (server.rb):

require "rinda/tuplespace"

port = 4000
ts = Rinda::TupleSpace.new
DRb.start_service("druby://:#{port}", ts)
puts "Rinda listening on #{DRb.uri}..."
DRb.thread.join

We start by setting the default port to 4000. We then create a new TupleSpace object. The call to start_service starts a local dRuby server listening on port 4000. Passing the TupleSpace as the second parameter sets the server’s front object to the TupleSpace. The URI of the server is output, and the program waits for a kill signal, and we have a Rinda server ready to be accessed by a client. To start the server:

$ ruby server.rb

Here’s a simple interactive client (client.rb). It allows input to be gathered on standard input, with the form .

require "rinda/rinda"
include Rinda

port = 4000
ts = DRbObject.new(nil, "druby://:#{port}")

while message = gets
  begin
    args = message.split(" ")
    method = args.shift.to_sym
    topic = args.shift.to_sym
    message = args.shift
    message = nil if message == "nil"
    tuple = [topic, message]

    puts ts.send(method, tuple)
  rescue Exception => e
    puts e.message
  end
end

We’re using the send method in order to invoke the method specified on standard input. There’s also a conversion between nil as a string into nil proper. This is necessary because of wildcards, which we’ll get to in a minute. With the server running, run the client to start a new interactive session:

$ ruby client.rb

Below is a sample session (output is indented for clarity):

write message hello
    Rinda::TupleEntry:0x4f998
write message world
    Rinda::TupleEntry:0x4cfa4
read message nil
    message
    hello
take message nil
    message
    hello
take message nil
    message
    world

The inputs above translate to these method calls:

ts.write([:message, "hello"]) => Rinda::TupleEntry:0x4f998
ts.write([:message, "world"]) => Rinda::TupleEntry:0x4cfa4
ts.read([:message, nil]) => "hello"
ts.take([:message, nil]) => "hello"
ts.take([:message, nil]) => "world"

Note that reads do not remove the tuple from the tuplespace. The tuple is left untouched such that other clients can access the same data. This is an important concept when dealing with synchronization that we’ll return to in a future post.

The structure of the patterns used in the read and take methods deserve some additional attention. When the tuplespace is queried through a read or a take, there are two criteria used to determine if a tuple matches. First, the length of the tuple must match the pattern’s length. In the example above, all patterns are pairs (2-tuples), so only tuples that are pairs will qualify. Second, the tuple must match the pattern specified. When specifying a pattern, nil is used as a wildcard, which will match anything.

Assume that our tuplespace has the following tuples:

1) [:message]
2) [:message, "hello"]
3) [:message, "hello", "world"]

If the pattern [nil] is specified, e.g. ts.read([nil]), only #1 will match.

If the pattern [:message, nil] is specified, only #2 will match. Of course, [:message, “hello”] would also result in #2 matching as well, while [:message, “world”] would not.

Using [:message, nil, nil], [:message, “hello”, nil], or [:message, nil, “world”] would all result in #3 matching. So it is important to remember that patterns are matched on both length and content of the tuple.

Conclusion

We haven’t even scratched the surface of what can be done with Rinda. This introduction only gets the simplest client and server talking to each other. Stay tuned for more.

After all the cards are written and estimated, it is time for the customers to pick the first iteration, for which they need a velocity and a length of time. Usually after a creatively exhausting meeting, both are chosen arbitrarily , or with minimal stimulus.

I have been on projects with one week iterations and one month iterations, all with varied success. The formula for the length seems to have different inputs for success, involving customer participation, speed of changing requirements, and physical location. All of these factors are prescriptive, attempting to plan the needs of the team based upon known constraints, something software craftsman are very familiar with. The most successful iteration length for the teams I have been on is one week, regardless of other constraints. Thanks to an excerpt from Donald Norman’s new book, Emotional Design, I think I understand why environmental constraints do not affect iterations ideal length.

“…being happy broadens the thought process and facilitates creative thinking. [Alice] Isen discovered that when people were asked to solve difficult problems, ones that required unusual “out of the box” thinking, they did much better when they had just been given a gift – not much of a gift, but just enough to make them feel good.”

One week iterations are the perfect length for all of these emotions to be useful. At the begging of an iteration is when you solve the tougher problems, not worried about a deadline. Your creativity can know no boundaries, everything is possible. A refactoring spanning a few days seems manageable, a large story looks like it can get done without sweating. Really, as a developer, the day after the iteration meeting is often the happiest day you have, especially if you hit your velocity. You are ready to spread your wings and impress the customers at the next iteration meeting.

The book goes on to say, “…when people are anxious they tend to narrow their thought process, concentrating upon aspects directly relevant to a problem.”

As the iteration progresses, you start to feel the meeting. You think “I can’t get all of my ideas done by the meeting” and “I can’t show up empty handed”. This is when some of the more grandiose ideas get cut and you start to concentrate more on the acceptance of stories. As it gets closer to iteration day, you become more granular, focusing all of your energy on acceptance of stories. This blocks out the bigger designs and system solutions from your frame of reference.

Even within a solution, I have banged my head against a solution for hours, without thinking to sidestep it. It is always harder to think of an alternative solution to a problem on the last day of an iteration than on the first.

This is good for iterations, you cycle through all of these emotions, not staying on any of them too long. Too many days at the beginning of an iteration leaves developers wide eyed about refactorings or experiments. Sprinting at the end of an iteration for too long leaves a team stressed and under productive. However, all those emotions in small doses, on a continuum is good for the project and good for the developer. You get to flex your developer muscles and try out something cool, but by the end of the iteration, you are focussed on finishing the features the customer needs.