8th Light Blog

Modules called, they want their integrity back

03 Feb 2012

TL; DR

Rather than including modules and hooking into .included, write a method that adds the behaviour (even if the method just includes and extends).

Introducing the pattern

It's common practice in Ruby these days to use modules for all kinds of crazy.

module IWishIWasAClass
  def self.included(klass)
    klass.send :include, InstanceMethods
    klass.extend  ClassMethods
  end
  # ...
end

Here is the most eye-widening example I've seen. You might think you're getting one thing, but you're not, you're getting all the things. Base, anyone?

So what's the problem?

Include has a meaning, and this changes its meaning.

When I include something, I expect to get the module's methods for my class' instances. Honestly, I know include is a method, but I think of it like a keyword. Until I realized this "pattern" was common, I was astounded when I would somehow magically have class methods, the class is a completely different object!

It pollutes the ancestry.

When you include a module, it creates a class with the module's methods and sets it as the superclass of the object. So first of all, it is pollutive (clutters up the ancestry when you are trying to look around) and second of all it is a performance hit. Whenever you invoke a method, it goes up the ancestry, one after the other, searching for the method in each of them. In this case, the module has no methods itself, it was just a way to include InstanceMethods and extend ClassMethods. An empty stop along the call chain. Even the Rails team has realized this and stopped using the InstanceMethods side of the pattern.

It's unreliable and breaks module semantics.

When I want the module's methods on my instances, I include. When I want them on my class, I extend. Thus these two should be equivalent:

class TheCommonWay
  extend SomeModule
end

class TheEquivalentButUglyWay
  class << self
    include SomeModule
  end
end

And they basically are equivalent, except if you start abusing hooks, consider:

module SomeModule
  def self.included(klass)
    puts "SomeModule was included in #{klass.inspect}"
  end
end

Class.new { include SomeModule }  # prints "SomeModule was included in #<Class:0x007fe17884f768>"
Object.new.extend SomeModule      # nothing printed

In the first example, the hook was invoked, in the second, it was not. So if you are relying on the hook to add the behaviour, then for it to behave correctly I must be aware of this and adjust my usage.

It's a problem of perception.

In his talk at GoGaRuCo Why does Jeff Casimir¹ write this code?

Because he wants to use it like this:

He knows he can extract the code with a module, so he has modules on his brain. And the way to use modules is to include them. I love around 7:19 when he says "I always use code before I write it. So I would write something like this where I would include the module". I consider that a best practice. But here, he isn't using the code before he writes it, it's already written in his brain. Which means the code that is supposed to use it (the code necessitating its existence) is written to the implementation he already has in mind.

This just uses modules to dispatch methods. There is absolutely no reason to include a module here. The module is mixed in because it will invoke the included method... which only needs to exist because that's how you hook into mixins. A regular method would work just fine. He could literally replace include Contact with Contact.included self.

So what is the alternative?

We need to stop cargo culting module patterns. We can get behaviour any way we want. In the above example, we could use this alternative:

def add_contact_associations_to(ar_base_subclass)
  ar_base_subclass.send :has_many, :phone_numbers
  ar_base_subclass.send :has_many, :email_addresses
end

class Person < ActiveRecord::Base
  add_contact_associations_to self
  def name
    [last_name, first_name].join
  end
end

class Company < ActiveRecord::Base
  add_contact_associations_to self
end

Now I'm not advocating toplevel method definition, in reality I'd probably put it in a module as a namespace. But this shows that the mixin was just an elaborate ruse to invoke a method that could be invoked directly, or replaced with something more expressive. Some say it's better to be clear over clever.

On Deject I just have a function with the same name as my module ². In one implementation, it extends the class you pass it. Sure you could get the same thing by just saying extend Deject but if it ever changes, then the interface will have to change, or I'll have to start abusing hooks. By having a method which is responsible for doing to the class whatever things need to be done, it does not change the meaning of include. In another implementation, it doesn't use modules at all, it just adds the desired methods.

When you need both class and instance methods, give your module a descriptive method which will do the extending and including (or simply defining), there is no need to rely on the hook.

Here are some example alternatives

Define the method explicitly

module M
  def m_ify(klass)
    add_class_method klass
    add_instance_method klass
  end

  def add_class_method(klass)
    def klass.class_method
      'some class method'
    end
  end

  def add_instance_method(klass)
    klass.class_eval do
      def instance_method
        'some instance method'
      end
    end
  end

  extend self
end

c = Class.new
M.m_ify c
c.class_method # => "some class method"
c.new.instance_method # => "some instance method"

Include and extend implicitly

module M
  module ClassMethods
    def class_method
      'some class method'
    end
  end

  def self.m_ify(klass)
    klass.extend ClassMethods
    klass.send :include, self
  end

  def instance_method
    'some instance method'
  end
end

c = Class.new
M.m_ify c
c.class_method # => "some class method"
c.new.instance_method # => "some instance method"

And of course, many things which modules are used for could actually be done with composition ;)

Footnotes

1 Jeff is a great guy and his talks always get me thinking and reconsidering, so I feel okay using him for my example. Though this is quite common across the Ruby community. Plus it's hard to resist when the name of the talk is "The Problem is Your Ruby" :P

2 In this example, "deject" is a verb so I felt okay making a method with the same name as the constant itself. I debated this a while, but ultimately was convinced by Execution in the Kingdom of Nouns that it was an acceptable decision.

Service Oriented Agony

01 Feb 2012

I sat down with a group of developers today to do a retrospective on a project. They told me that project management has been complaining about their velocity. It wasn’t serious, but the developers felt bad because things do seem to take longer than they should. When I asked them why, they said that the system was very complex, and making changes to it was time consuming.

I asked them to draw the system on the whiteboard. Here’s what they drew.

The system has several front end processes, each is its own Rails app. They talk to each other by throwing URLs at each other, and loading cookies, etc. Pretty normal.

Each of the front ends talks to some gems which act as interfaces for the services. The services are individual Rails apps that communicate through HTTP using JSON. These services each manage a backend database.

I suspected the answer, but I asked the developers why this structure slowed them down. They told me that every time there is a new feature to add, they have to add it in three different places. They have to change the front end to add the UI gestures. They have to add a new function to one of the gems in order to provide the UI with the new service interface. Then they have to add a new MVC triplet to one of the services that the gem will use to access the data.

I asked them if this was true even if there weren’t any schema changes; and they told me that it was because the gems and the services had evolved with the whole application, and only had functions for the current features. New features often require new functions even though there’s no schema change.

This is a pretty typical structure. I’ve seen it many times before. I’ve seen it in Java apps, C++ apps, and now even Rails apps. The structure seems obvious to system designers who have grown tired of single monolithic systems and want to break those systems up into components and services. What could be more natural than to break the system along the lines of data base managment?

Unfortunately this is a huge violation of the Single Responsibility Principle — or its big brother the Common Closure Principle. These principles tell us to group together things that change together, and keep apart things that change for different reasons. Unfortunately the above design separates things that change for the same reasons, and groups together things that change for different reasons. No wonder the developers feel like they are going slow!

When you separate things that change for the same reasons, you have to make changes in many different places in the system. If those places are in different applications, then you have to switch mental contexts for each change. Testing the change is hard because you have to have all the components running to test it end-to-end. And you have to make sure that all the interface points work as desired. So it’s a lot of work just to get anything working.

Moreover, when you group together things that change for different reasons, you expose the components of the system to collateral damage, thrashing, CM collisions, and a whole host of other problems. Changing one of those gems, for example, has an impact on all the front ends, even though some of those front ends won’t use the part of the gem that was changed.

So what’s the solution? First of all, I question whether the system needed to be partitioned into services. Services are expensive and complicated, you should only create them if you absolutely need to. It’s always easier to live in a single process. Remember Martin Fowler’s first law of distributed objects: Don’t distribute your objects.

Let’s assume, however, that the system did need to be partitioned into services. Are these the right services? Dividing a system into UI and DB services is a physical partitioning. What we should be looking for is families of business rules that are separate from each other, and can be partitioned into true business services.

Finally, let’s assume that the physical partitioning was necessary for some reason. Then we need to find a way to make the interfaces of the gems and DB services generic so that every new feature doesn’t require a change throughout the whole system.

Like I said, I see this kind of partitioning a lot; and it always has the same outcome. It slows development because it smears features through many different layers. Many systems could be streamlined, and development made much faster, if the system designers paid more attention to the Single Responsibility Principle.

The Ruby Colored Box

31 Jan 2012

I recently read a blog about a guy who had a bad interview. The blog was well written and pretty funny. He metaphorically described the interview in terms of a mexican chef being grilled about how to make crème brûlée. He made a bunch of interesting points about how unfair the interview was, and how you shouldn’t ask a mexican chef how to cook pastry.

The blog was in two parts, the mexican chef metaphor, and then a rant. By the time I got to the end of the metaphor I was pretty much thinking that the author was a bit of a whiner. OK, so the interview was unfair. What interview isn’t? Why would any employer want to conduct a “fair” interview. I’m certainly not fair when I interview people. I put them under all kinds of stresses, and throw lots of strange issues at them. I want to see how they behave when confronted with unfamiliar problems. I want to know if they can extrapolate their experience into a new domain.

But I also understood his point. I’ve been to interviews where I’ve bombed, and can remember thinking they were unfair at the time. They weren’t, it’s just that interviewers aren’t trying to find the first person who might fit. They’re trying to find the one person who stands out. That means they’re going to pass over a lot of good people. That’s just the breaks of the game.

But then I got to the rant, and I read this:

My attitude is if I’m a good Ruby programmer, and you’re trying to hire me when the supply for Ruby programmers is low and demand is high, that before you even talk to me you’ve spent at least 10 minutes Googling for my name, looking at my code, and figuring out who I am, rather than spending an hour subjecting me to a series of ad hoc programming questions in areas I may or may not specialize in. That 10 minutes of Google will tell you a lot more than asking me to come in and scribble stuff on a whiteboard.

I think this process has left me a bit more discerning about the companies I’ll actually interview with. When you’re trying to hire talented developers in a scarce market, please do your due diligence and don’t insult somebody skilled by asking them to do a degrading whiteboard interview instead of looking at code they have freely available on the Internet or just looking over their shoulder as they code on a computer, preferably their own, at least the first time you get to know them. You may even learn something.

The last thing I want is to hire someone who thinks I owe them a job. Does this guy really think he’s so hot that I’m going to do research on him before I meet him? Am I supposed to kneel down and kiss his sandals for granting me an audience? Just who does he think he is?

Oh! He’s a Ruby Programmer! That explains it! Of course I should kiss his sandals. He’s a Ruby Programmer. He’s on GitHub. Oooooohhhh!.

Dear Employers in search of Ruby Programmers:

You don’t need Ruby Programmers! You just need programmers! Any decent programmer can learn Ruby in a week. And Rails only takes a little longer. Don’t hire these prima donnas who think they’re God’s gift to Ruby shops. They’re not. They’re just generally inexperienced kids who happened to learn a pretty easy language and a pretty easy framework.

Instead, find some programmers who have been around the block a few times. They should know multiple languages, and be able to deal with many different and novel situations. Don’t worry if they don’t know Ruby, they’ll know it, and Rails, within a month. If you like you can temporarily hire some super-expensive consultant to teach them if you like. They’ll learn quickly, with or without.

What you are looking for, if I may drop back into the metaphor, are mexican chefs who don’t panic when you ask them how to prepare crème brûlée. It’s ok if they don’t know (though don’t let them know that!). What you are looking for is whether or not they can extrapolate from their experience. In other words, to drop back out of the metaphor, if you are interviewing a Ruby programmer, your first questions to him should not be about Ruby.

Ask him something outside his comfort zone, and see if he can extrapolate. Ask him, to write The Sieve of Eratosthenes in C. If he doesn’t know C, give him a copy of K&R and come back in 20 min. He won’t have the right answer; but that’s not what you are looking for. You are looking to see whether or not he has the experience and resourcefulness to fruitfully attack the problem. You are looking to see if he can think outside the Ruby colored box.

Effective Customer Consultant Relationships

20 Jan 2012

As a customer, how can you get the most out of your consultants? As a consultant, how can you deliver what your customers really want? I have been on both sides of this relationship and experienced both successful and unsuccessful results. In this post, I will go through the typical aspects of these types of relationships and identify action items that I have used to forge effective and lasting customer consultant relationships for both parties.

The Typical Process

The process I am about to outline is not the process for all software consulting firms. However I have seen this happen frequently and have run into this situation myself as a client and as a consultant.

The customer comes up with a general idea of an application that would bring value to a group of people. Then he/she finds a firm that has experience building software and hires them to build the application.

The consulting firm takes a week or so to build a master plan based off of the customers ideas and shares it with the customer. Once the customer okays the plan, the consulting firm starts building the application. Since applications are very complex, the firm builds the application assuming basic details like user flow, page layouts, etc.

After the firm has done a week or two of work, the customer has had more time to think about how their app should work. When they see what the firm has created so far, it doesn't quite match up with what the customer has envisioned in their head. So they give the consulting firm more specific information about their application. With the new specific information, the consulting firm fixes their improper guesses and adjusts their code to line up with the new information the customer has provided. The fixes took time away from development, so now the project is a little rushed.

A couple weeks or months pass, and customer has come up with new ideas that will add tons of value to the application. So they share these ideas with their consulting firm. The consulting firm informs the customer that making these changes will add significantly to the development time because the system wasn't designed to support these new features. The customer is not happy to hear this, but this new idea is really important, so it has to go in. Now the project is over budget, over time, and rushed because the consulting firm has to change the structure of the application to accommodate the new idea.

This process occurs because the business requirements of a project evolve over time as the customer learns what features add value to the application. In this scenario, whenever a business requirement evolves, the consulting firm has to backpedal to add it into the system.

How To Avoid Major Delays

A major fear customers have is that the project will take longer to develop than the original estimate suggested. At the same time, one of the hardest things to do as a consultant is to estimate how long a piece of software will take to develop. To minimize the variance in estimations, it is imperative that the development schedule be broken down into a prioritized series of fully-thought out minimum marketable features (MMF). Also, initially only the most important MMFs should be estimated and built out to form a minimum viable product (MVP). Additional features can be added on top of the app once market feedback has come in from the MVP.

A MMF is defined as the smallest group of functionality that adds market value to an application. When a project's functionality gets broken down into MMFs, the project gets built in small working slices. Each slice can be demoed once it is done, and feedback is easier to give when the customer can play with a demo.

A MVP is defined as the smallest group of MMFs that allow the application to be deployed. As the time and scope of a project gets larger, so does the amount of complexity. No matter how experienced a consultant is, he/she won't be able to clearly picture what the application will look like until relevant parts of it get developed. Therefore, it gets harder to estimate features as the project specs list get larger.

Additionally, a tight feedback loop should be established. The customer's specialty is understanding the business requirements and the consultant's specialty is implementing functionality. Since the business requirements change over time, the customer and consultant have to work very closely to ensure that the consultant is implementing features that fulfill the current business requirements.

Customer Action Items

Demand that your software gets built in short release cycles

You are more capable of giving quality feedback when you can play with a demo of your software (plus playing with your software is fun). Additionally, your consultants will spend less time guessing and more time developing when you can give great feedback every week.

Break apart your idea into minimum marketable features

When you create a MMF, it is easier to drill down into the specifics of what you want out of a certain aspect of your application. Additionally, it makes it easy for your contractors to build in short release cycles when they can simply implement a handful of MMFs per cycle.

Initially, only build enough features to create a minimum viable product

You won't know what actually brings value to your market until you can release something. When you put a MVP in a client's hands, they will be capable of giving you extremely valuable feedback that you couldn't get otherwise.

Be available all workday long for questions that your consultants come up with

It is more cost effective for you to answer questions that your consultants have, rather than them guessing an answer and you having to correct their mistake down the line.

Consultant Action Items

Build software in short release cycles

Building software this way creates a tight feedback loop with your customer. Also, it pushes you to build an extensible architecture that can deliver business value amidst changing business requirements.

Persuade your customer to break up their idea into minimum marketable features

If they don't provide a MMF with specific acceptance criteria, then it will be extremely difficult for you to deliver value to them (unless you can read minds).

Don't estimate features that won't get implemented in the upcoming release cycle

No matter how smart you are today, you will always be smarter tomorrow. So don't waste your time estimating a feature that won't get implemented in the near future. Your estimation will change as you learn more about how the system should work.

Communicate with your client on a daily basis

Even if you only communicate with your client 5 minutes a day (excluding weekly product demos/meetings), your customer will feel in control of their project if he/she knows what is happening every day. Additionally, the more feedback you get from the customer, the less guessing/mind-reading you have to do.

Conclusion

It is both the customer's and the consultant's responsibility to forge an effective relationship. Both parties should work together with an understanding that the business requirements WILL evolve. If they follow the action items that I outlined, their relationship will be extremely effective at building projects that produce value.

Fecophiles

20 Jan 2012

→
I got an interesting email yesterday. It contained the following paragraph describing an email he had sent to his co-workers about a refactoring he had done:

When I originally sent this email to my co-workers, I got the following reaction; 1 team member thought it was better & 2 team members thought it was harder to understand. Of the 2 who thought it was worse; 1 thinks I should change it back, and the other is willing to put up with my change to shut me up :-)

Here’s the letter he sent to them, showing them the before and after of the code he had refactored.

Subject: FW: Refactoring of the day

 
Am I missing something?  Or did I just refactor a 31 loc method to 2 loc?
 
------- before ------
public static string GetLtsCode(IventoryBinItem item)
{
    string ltsCode = null;
    if (!string.IsNullOrEmpty(item.ParentItemNo)) //child item
    {

        ltsCode = "PzT";
        var isNfoOrDisc = item.IsNfoItem || item.IsDiscontinuedItem;
        //if (isNfoOrDisc && item.ParentIsNfoOrDiscontinuedItem ||
        //    !isNfoOrDisc && !item.ParentIsNfoOrDiscontinuedItem)
        if (!item.ParentIsNfoOrDiscontinuedItem)
        {
            ltsCode = "PzT";
        }
        //else if (item.ParentIsNfoOrDiscontinuedItem && !isNfoOrDisc)
        else if (item.ParentIsNfoOrDiscontinuedItem) //always capture demand for children of nfo (april19 change)
        //and childitem is not, then mark it as regular, otehrwise both PzT
        {
            ltsCode = null;
        }
    }
    else //parent
    {
        ltsCode = null;
        if (item.IsNfoItem || item.IsDiscontinuedItem)
        {
            //april 19 change
            //ltsCode = "PzT";
            ltsCode = null;
        }

    }

    return ltsCode;
}
-----------------
 
---- after ------
public static string GetLtsCode(IventoryBinItem item)
{
    bool isPzT = item != null
              && !item.ParentItemNo.IsNullEmptyOrWhiteSpace()
              && !item.ParentIsNfoOrDiscontinuedItem;
    return isPzT ? "PzT" : null;
}
------------------
  
---- And later refactored to ----
public static string GetLtsCode(IventoryBinItem item)
{
    return IsPzT(item) ? "PzT" : null;
}

private static bool IsPzT(IventoryBinItem item)
{
    return item != null
       && !item.ParentItemNo.IsNullEmptyOrWhiteSpace()
       && !item.ParentIsNfoOrDiscontinuedItem;
}
------------------

Now let me be clear. The first code is crap. Major crap. I mean it’s a really stinky brown and greasy pile of poop. It looks like it came out of a very sick dog.

I was astounded that anyone might think that crap was better than the rather nice refactoring. I guess some people just like to smell poop. I call them Fecophiles.

I thought that perhaps there was something wrong with their noses. After all, if you live in crap, you might just stop smelling it after awhile. So I determined to shove their noses in the fetid pile by investigating the smell in all it’s rich and pungent detail. So I sent the following letter back to my friend.

Let’s just walk through that previous code step by step and really smell it!

We start out buy initializing ltsCode to null. Fine.
Then we encounter a double negative (if not null/empty) and an indirection (item.ParentItemNo). We need a comment to understand it. Apparently if the ParentItemNo of the item is null or empty then it means that the item has no parent. If we negate that, then the if statement is trying to say “if this is a child” The comment tries to point this out, but the grammar is bad. It would be nicer if the if statement were if(isChild(item)) or if (item.isChild()).
In the body of the if statement we are a child, so we set the ltsCode to PzT.
Ignore commented out code.
Set the var isNfoOrDisc to an interesting expression. Oddly, I can’t see any place where that variable is used. It used to be used, in the commented out code, so maybe someone just forgot to comment out that var too, eh?
A negative if statement (it’s easy to miss that bang).
The if statement checks whether the parent is not nfo or discontinued. If so, sets ltsCode to PzT if . But ltsCode is already PzT. Maybe someone forgot to comment out this whole if statement too…. <bah!>
More commented out code. ugh.
Ah! Now in the else clause of the “parent Not nfo or Discontinue” if statement we check the boolean opposite. Does this programmer not know how if/else works? Why in hell is he checking a tautology. We already KNOW, in the else clause, that the parent is nfo or Discontinued. Sheesh.
And in that else clause, we set ltsCode to null. Wait… Wasn’t it already null?
(pant, pant, pant) ok, we’ve reached an else. What else is it? It’s the else from the double negative if statement in point 2 above. Ah, so this is where we have an item with no parent. That //parent comment is strange. Does it mean that the item is a parent? How would we know that. All we know is that the item does not have non-null or empty parent item number. So what’s with the //parent comment? It’s a lie, or at least a wild mistruth. I think that comment needs to be commented out. (grrrr).
We set the ltsCode to null. Hmmm. Let’s see. What was the ltsCode be before we set it to null? Why, I believe it must be null already! Is there any pathway through this horrible rats nest of code that might set ltsCode to something other than null at this point? No!
and now the coup-de-gras! Another if statement that checks whether the item is nfo or Disconnected. And what does this if statement do if that’s so? Why, it sets the ltsCode to … NULL. Of course! NULL. Brilliant!

Summary. This code is a mess. A whole-hearted unmitigated indisputable mess. It’s full of inaccurate and ambiguous comments, useless variables, if statements that have no purpose, and some truly convoluted logic. And what was the goal? The goal was:

return PzT if the item exists and has a parent that is not nfo or Disconnected. Otherwise return null.

OR, to put it in simple code:

	
 public static string GetLtsCode(IventoryBinItem item) {
  return IsPzT(item) ? "PzT" : null;
 }

 private static bool IsPzT(IventoryBinItem item) {
  return item != null
    && item.HasParent()
    && item.ParentIsNotnfoOrDiscontinuedItem;
 }

I am ashamed of anyone who thinks the original code is simpler and easier to understand.

I don’t know if they got the message. But even if they did, and they could suddenly smell their home for the first time, it’s a drop in the bucket. There are lots and lots of fecophiles out there who need to take a step back and inhale deeply through their noses.

8 Things You Ought to Know If You Do Not Know Anything About Hiring A Software Developer

19 Jan 2012

“The one absolutely solid place to store your capital today — if you know how to do it – is in software developers’ wallets.”

...is what Venkatesh Rao states in his thought provoking article “The Rise in Developeronomics” published Dec 5, 2011 on Forbes. He argues that the future of our economy will be heavily dependent upon access to software development services as starting, growing, and running your business is bound to involve software nowadays. So he makes the case that the best friend to have to face these challenges is... a software developer.

I did not come to this industry with a software background. I studied math, physics, business, then finance. But over the past 12 years or so, I have learned quite a bit about how to pick a software... “friend”. So let me share with you 8 things I would want to know if I was starting afresh.

I would ask about the development process. I seem to take it for granted that all software shops operate on some type of “Agile” development methodology. The reality is much different. Beware of a once favorite software development process named the “Waterfall Model”, a sequential model in which steps are completed one after the other. First the customer would specify ALL requirements, then the app would be designed and coded, then -- and lots of time is elapsing here as you’d progress thru the steps -- it would be tested; then deployed. The drawbacks of this process is that it’s lengthy, unresponsive to change and yes, super costly. So the process you want your software friend to abide by is an Agile one. Because an Agile process calls for short development “iterations”, you get value in short increments of time. In a nutshell, it’s a collaborative process that delivers quickly and enables you to adapt to change.

I would ask about the development practices. Just like you would expect of your accountant to abide by certain Generally Accepted Accounting Principles, you ought to expect that your software “friend” abide by the following practices.

The first one on my list would be TDD: Test Driven Development. No later than yesterday, I produced a spreadsheet that summarizes how an amount of money gets allocated amongst different accounts. In doing so, one could easily forget to include a cell. To put TDD in perspective to the non developer, my test to avoid such a situation would be to have a cell in which the formula tests that the sum of the parts would equal the whole, before even starting the allocation. A good software friend practices that as a reflex. Before writing any code, he or she writes tests to ensure that the code to be written will produce the desired outcome.
The second one would be pair programming. If you have ever looked even remotely at code, it’s a lot of text. To me it may be mostly gibberish, but it does not take much convincing to understand that two sets of eyes are better than one. So hopefully, your friend has a friend, because, that will ensure less bugs.
The third one is those short iterations we talked about earlier.
The fourth one is continuous integration. This means that your friend is diligent about integrating to the rest of the system as a whole. It ensures that everybody’s work works together.

I would ask how my potential software “friend” would keep his or hers claws sharp. Again, just as you would expect your accountant to be aware of the changes in tax laws, you should expect your software friend to be aware of the changes in the software industry. My software friends usually go to conferences, read books, reflect on their work whether through blogging or presenting to others, and meet with other fellows in the field. They know several languages and work across multiple platforms. Better yet, they fear not learning a new one. Above all, they are far from being the stereotypical isolated geek one could envision.

I would ask for proof of talent. Although a degree from an ivy league university might do the trick for some, quality experience does set the good developer apart. My developer friends have loads to show to testify for their talent. For some, from past work done, for others, from apprenticeship projects, but for all, from hands on development.

I would ask about how they estimate. Systems built nowadays have variable scope. As a result, it can be difficult to estimate how long the project will take. However, I would advise you to befriend a developer that gives you a reasonable estimate. An estimate that could provide you with a bracket of possible values: one pessimistic, one optimistic, and one realistic. I may even push it to ask with what confidence do past project estimations fall within a 95% accuracy within the aforementioned brackets.

I would ask how deadlines are met. Deadlines are a critical variable for any business. If I were to befriend a software developer I would expect to hear that since the development process relies on short iterations and prioritization of requirements, I (the customer) hold the key to deadlines. I (the customer) would be the one prioritizing which features get attended first. I (the customer) would hence have control on where my ROI would come from.

I would ask about cost. Of the upmost importance, staying within one’s budget is a goal that short iterations can help to fulfill. When getting value in short iterations, cost is easy to tame. Prioritization ensures that the most financially critical decisions stay within the hands of the customer. A good software development friend will bill at a certain frequency of iteration completions.

I would ask about change. I would say to my potential friend: “How will my app be able to evolve overtime?” -- deeply concerned with the ever changing technologies, market requirements, and products I could offer. I would expect him to state that he values feedback and collaboration. A good software developer friend embraces change in requirements which are bound to happen overtime. Oh, and do I need to stress again that a good software friend develops in short iterations, which by itself, guarantees an adaptable progress?

The Letter

12 Jan 2012

We need to become a self-regulating and self-policing profession. The stakes are simply too high to allow software to remain in the current ad-hoc limbo of hackers, heroics, and hermits.

Consider how much software you interact with every day. Your alarm clock, your cell phone, the television and cable box, the remote control, your toaster oven, your watch, your car, the train to work, the cash register at Starbucks, the coffee maker at Starbucks, the elevator you ride, etc. etc. The list is virtually endless. Nearly every aspect of our daily lives, nearly ever corner of our civilization is somehow touched, controlled, managed, or influenced by software.

Think about that again. Virtually every aspect of our lives has a software component; and yet we exert absolutely no regulatory control over that writing of that software. Any Harry Hacker with a “J” in his name can get hired to write Java code. And the strongest likelihood is that Harry J. Hacker’s code be crap, will be wrong, and will not be explicitly tested before it is shipped.

Plumbers are regulated. Electricians are regulated. Architects, lawyers, and doctors are regulated. Why aren’t we? Don’t get me wrong, I don’t want government to be the regulator; I want us to self-regulate. But if we software developers don’t figure out how to do that, then government will certainly step in. And then life will get really bad.

Yesterday I received a very scary, letter that underscores this point rather dramatically. I thought you’d like to read it. I worked with the developer to sanitize it so that the innocent people in this story are not punished. I regret that I can’t splash the guilty parties’ names all over twitter though.

Hello “Uncle Bob”,

I’m a 34 year old freelance programmer who has been developing software for 15 years.

Some time ago I was hired as a team leader for a safety critical embedded system that controlled a medical surgery device. Everything was fine during the first months. I put a great deal of architectural effort into safety. I used Active-Objects for safer threading, UML-Generated State-Machines for stateful safety-checks, simulation, and lots of Unit-Testing (though not with full coverage).

Oh, things weren’t perfect. We had been asked to finish a four year project in less than 1 year; so time pressure was very high. Even so I had a good feeling that the device would be safe for millions of treatments.

Eventually it became clear that we could not deliver the full release on time. When we told our manager, he began to worry about his bonus. (At least that’s my personal suspicion). When I told him we couldn’t meet the schedule, he told me I was not allowed to make any more estimates. He shouted a lot all day long and forced everyone to work 7 days a week. He was in such a rage that he even scared the validation team into concluding the official medical validation after just a few days; which was far too early.

He knew that safety was the most important issue for me, so he began to cut my responsibilities. He eventually gave full control of the project to the
youngest and most impressionable programmer on my team. I continued to code in the main-branch of the project; so the manager made his own code branch and, together with the rookie, produced his own version of the software.

So I terminated my contract with the company.

A few days later the first bug occurred during a human trial. Fortunately, during the early days of the project, I had made the software robust enough that the bug didn’t harm the patient. The device just stopped operating before starting the automated surgery. You’d think that would have been a wakeup call, but the company didn’t even analyze the bug (not to mention stopping the unstable product). More bugs were found later. Even obvious bugs like mixing up the surgery directions (upwards/downwards) were found during the first human treatments. It is a real possibility, and one of my great fears, that tomorrow some patient will become severely disabled.

The other programmers didn’t quit. They told me: “the boss is the boss – we just do what he tells us – it’s his responsibility.”. So I figured I must have been crazy for quitting. I felt weak and worthless, like I couldn’t handle the pressure. But lately I started reading “Clean Coder”. And it’s made me consider that, perhaps, I’m not weak – but the opposite: strong.

I am, of course, furious about the manager who drove his organization to such unprofessional depths. He is an idiot and a criminal, and I hope he winds up in prison. But I am even angrier at the developers. Not only were they accomplices to that criminal idiocy; they made the one guy who took a stand feel stupid and weak. They are the ones who are stupid and weak. They should not be programmers. Programmers are better than that.

Aren’t we?

Flipping the Bit

11 Jan 2012

There’s a bit you need to flip in your head. It’s just one bit. In this blog we’ll call it “THE BIT”. Some of you have already flipped THE BIT. The rest of you need to flip THE BIT as soon as possible.

Once you flip THE BIT your life will be easier, your code will be cleaner, you’ll get done with software faster, your software will work better, and fuzzy bunnies will happily hop over hillsides overarched by double rainbows all the way!

Before I tell you what THE BIT is, let me show you an example of an article produced without the benefit of THE BIT being flipped. The article is here, and was written by someone named Tom Fischer. As I write this blog, Fischer’s article has been read eleven thousand times. That’s eleven thousand people who’ve been influenced by the wrong polarity of THE BIT.

Fischer’s article makes the point that unit testing isn’t always necessary. He uses graphs and charts with clever little colorings to support his arguments. Those arguments are two-fold.

Unit testing takes time and can delay projects causing them to lose money due to lost opportunity¹.
Unit tests don’t find all bugs because many bugs are integration bugs, not bugs in the unit-tested components.

Fischer goes on to say that unit tests are great and can solve lots of problems, etc, etc, blah, blah. But then he erases all those positive and friendly words by concluding with this:

We can no longer rely on a general acceptance of the myth that unit testing is a universal panacea, but need to focus unit testing on aspects of development where it is most effective, and be prepared to actively justify its use.

Oh my goodness! We have to be prepared to actively justify unit testing? Be warned! The Unit Test Compliance Squads will be making the rounds soon. Anyone who cannot actively justify their unit tests will be duly punished! <samuri scowl>

Who’s going to write unit tests if we have to actively justify their use? Who’d be fool enough to take that risk? So we can summarize Mr. Fischer’s article with the following sentence: “Don’t you dare write unit tests unless you KNOW, and can actively justify, that you have enough time.”

So the question is: Who’s got enough time? Who, out there, has schedules that are so forgiving that you can actively justify the risk of writing unit tests? Who out there is working in an environment where opportunity costs and time-to-market aren’t an issue? Who out there has a cushy-dreamy-stress-free-unicorn-riding project in which you can safely absorb the terrible cost of unit tests? Answer: Unicorns and fuzzy bunnies. But no humans that I am aware of.

Therefore, the only real conclusion of Mr. Fischer’s article is that nobody should ever write unit tests no matter how beneficial they might sometimes be, because the risk and cost is so great that you must actively justify their use. (Note, that there seems to be no need to actively justify their lack…)

Mr Fischer has not flipped THE BIT in his brain. He still thinks that unit tests take time. Apparently he thinks they take a lot of time. He still thinks that there are situations in which unit tests aren’t helpful. He hasn’t flipped THE BIT. What planet is he living on?

THE BIT

What is THE BIT? THE BIT is the boolean variable within your subconscious that represents your belief that unit tests take time. I want you to flip that bit so that your belief changes. I want you to believe that Test Driven Development saves time in every case and every situation without exception amen.

Notice that I did not say: “Unit tests save time”. I said: “TDD saves time”. I think that’s an important distinction. I don’t want you believing that unit tests written after the fact are anywhere near as beneficial as the discipline of TDD.

Here’s the test for THE BIT. You can have a friend ask you these questions. If you answer them as shown below, you’ll know that THE BIT has been flipped:

Given a task, will you finish faster using TDD?: YES.
Are there any tasks that you can finish faster without using TDD?: NO.
I understand that TDD might help in the long term, but what if the job is really short term? Would you still use TDD?: Yes, because TDD is faster even in the shortest term.
What if the schedule is really tight and the boss is breathing down your neck, would you still use TDD?: YES.
In every case?: YES.
Is there any case where you would not use TDD: NO.
What if you were on the Starship Enterprise and the warp coil was seconds away from an anti-matter explosion and all you needed to do was invert one IF statement to save the ship. Would you use TDD for that?: Yes.
Why?: Because I’d get done faster.
Even for just one IF statement?: Yes, even for just one IF statement.
Do you mean to tell me that there is no case, no case at all, in which you would not use TDD?: Well, I might not use TDD if I needed the task to take a really long time to finish, and have lots of bugs. But other than that? No, no case at all.
What about GUIs?: Ah, GUIs!²

That’s THE BIT. When you have flipped THE BIT in your head, then when your boss tells you to stop writing tests because you don’t have time, you’ll refuse and tell him that without TDD it will take you longer to get done — and you’ll believe it! The more the pressure is on, the more you’ll rely on the TDD discipline.

How do you flip THE BIT? I can tell you how I flipped my BIT. I used TDD for a month or so; and my BIT flipped. That’s all it took for me — just doing it. Maybe that’ll be enough for you.

¹ It seems to me that you can lose a lot of money by shipping code that doesn’t work too, but — oh well.

² We’ll talk about GUIs in more detail later. The short answer is: Yes. The medium answer is: Yes, for all the dynamics.

Hello iOS Development

19 Dec 2011

When first getting started with iOS development, there were very little resources to get up and started. And with the strict NDA Apple had over the SDK, there were actually very little resources for any level of iOS development. The goal of this tutorial is to walk you through building a simple app demonstrating the use of actions, outlets and controllers.

What you need to get started

An Intel based Mac
Xcode 4 with the iOS SDK
A basic understanding of Objective C
An iOS Dev Center account (For testing apps on real devices and App Store Submission. Not needed for this tutorial.)

Your First Project

Launch Xcode

Xcode is the IDE you will be using to development applications for an iPhone or iPad. It comes with a simulator the two devices. When you start up Xcode, one of the first things you will probably want to do is create a new project.
create a new xcode project

Here you are given a choice from different types of applications. At the bottom of the screen there will be a short description of the selected application template, but here are some examples of different uses. Navigation-based Applications would be used for Mail-like apps, Tab Bar Applications would be used for iTunes-like apps, and Split View-based Applications would be used for Mail-like apps but for the iPad.

For this tutorial, we will be using the View-based Application.
Choose View-based Application

Name the project "HelloWorld" and choose a location to save it. Don't worry about including unit tests.

After creating the app, you will be shown a screen which is split into many different panes. If any of the panes are hidden, there are buttons at the top-right of Xcode to toggle their display.

The left hand pane is the Navigator, which has multiple purposes. Most commonly, it will be used to display your files arranged into groups (which do not correspond to directories on the filesystem). It can also be set to show your compile errors or logs.

The bottom pane is the Debug Area, which will show log statements and exceptions.

The right pane is the Utilities Area, which is primarily useful when editing xib files.

The middle section is the main section and is where the current file will be displayed.
Edit HelloWorldViewController.xib

The gray box you are presented with is a WYSIWYG editor for your main view. Drag a Label from the bottom of the Utilities pane onto the view and edit the text of the label. If you don't see an option for Label in the list of elements, you might have to switch to a different tab on the Utilities pane.

You should also take this opportunity to make the label wider. This will be important later in the tutorial.

At this point, you could run the application by going to the Product Menu and choosing Run and be presented with the text of your label, but we aren't done yet!

Edit HelloWorldViewController.h

What we are going to do here is to define two properties. One UILabel and one UITextField. You may notice the use of IBAction and IBOutlet in the method and property declarations.

IBActions and IBOutlets

The "IB" stands for Interface Builder. An action is simply an event triggered by the user action, e.g. a button being pressed. An outlet is an object in the view that is altered by a controller. Labels are a common outlet, but even objects like buttons can be wired as outlets. An example of this would be a on/off button, where you would toggle the text of the button between clicks.

The IBOutlet in the property declaration is a way to make the instance variables available to you in the xib file. That way, when you make changes to label or text field in your code, it will show those changes in the view. The IBAction in the method declaration where the return type would normally is not actually a return type, but instead makes this method available in the xib file as a target for an event. The sender argument is an optional argument and will be the object that initiated the event. In our case, it would be the instance of the UIButton that the user pressed.

You might receive some warnings until the next step is finished. This is because we've defined methods in our header file but have not implemented them.

         1         #import <UIKit/UIKit.h>
 2 
 3         @interface HelloWorldViewController : UIViewController {
 4             UILabel* helloWorldLabel;
 5             UITextField* nameTextField;
 6         }
 7         @property (nonatomic, retain) IBOutlet UILabel* helloWorldLabel;
 8         @property (nonatomic, retain) IBOutlet UITextField* nameTextField;
 9 
10         - (IBAction) sayHello:(id)sender;
11 
12         @end
13

Edit HelloWorldViewController.m

Here we are synthesizing our properties and implementing our action, which returns nothing. The sayHello action is doing a number of things in order. It's taking the inputted name from the UITextField, prepending it with "Hello " and outputting that result to the UILabel.

At this point, you should be able to compile your application without any warnings.

         1         @implementation HelloWorldViewController
 2         
 3         @synthesize helloWorldLabel, nameTextField;
 4 
 5         - (IBAction) sayHello:(id)sender {
 6             NSString* name = [NSString stringWithFormat: @"Hello %@", self.nameTextField.text];
 7             self.helloWorldLabel.text = name;
 8         }
 9         
10         // Remaining code below...
11

Back to HelloWorldViewController.xib

Our first version of this file only had a UILabel, so let's go ahead and add a Round Rect Button and Text Field to our view. If we were to run the app now, we would see all of our UI elements, but wouldn't do anything, because we haven't wired them to our controller.
Wire Up The Outlets and Actions

This is where we link our UILabel and UITextField in the controller to their corresponding objects in the view. Labels in the view correspond to instances of UILabel in the controller, Text Fields correspond to instances of UITextField in the controller, etc.

To see a list of available actions and outlets, right-click on the yellow cube, aka the "File's Owner." The File's Owner is simply the object that loaded up the xib file. In this case, it will be the HelloWorldViewController. Once you are in that menu, to wire up the label, drag from the circle to the right of helloWorldLabel to the label in the view. Do the same for your nameTextField and the Text Field in the view.

Wiring the action is a bit trickier but starts the same, with dragging the circle to the right of sayHello to the Round Rect Button. The only difference is that at the end of dragging the action to the button, you will be presented with a different menu of the possible events that the UIButton can trigger. For UIButtons, the most common event will be "Touch Up Inside." This event is fired when the user lifts their finger up inside of a button after that user pressed down the button.
Run It!

At this point, you should have a working app that allows you to type in a person's name and output it into a label. A github repo with a working example can be found here.

Fixed Feature

16 Dec 2011

Time and materials vs. fixed bid is the classic duo of billing models in software services business. Time and materials puts all the pressure on the client to make sure the work is completed. They must have enough control and transparency to be satisfied the work is moving at the right pace and in the right direction. Fixed bid projects put all the pressure on the services company to make their estimates for the whole project correct up front. Since software is a craft with a close problem finding to problem solving loop, we can never anticipate what lies beneath a given problem set, making fix bid a high risk.

8th Light uses a compensation model we call fixed feature which finds a middle ground between hourly and fixed bid compensation. Fixed Feature is an Agile method of fixed bid compensation that is unique in that clients only pay for the product they receive. At the beginning of an iteration we estimate using the PERT estimating technique. After estimating we make a commitment to those estimates and use a predetermined price per point to calculate the cost. At the end of the iteration we bill for the features that are accepted by the customer.

Because of this, clients do not have any obligation to manage personnel or hours, they get to focus on software features. Clients also know from the start exactly how much each feature will cost, regardless of the labor that goes into making it. The metrics collected can be very important to the client specifics about where the cost of overall software is actually going. From there they can iterate and effectively choose what features are most important to their business.

This payment models didn't come without pain. It made it transparent that our estimates were not very accurate. We had to learn how to be really good at estimating features, which is described in the white paper From Estimate To Commitment.