8th Light Blog

I find while writing web applications that I end up redefining what it means to go “back.” This means where I want the user to go when they hit the back button. The creation of AJAX has killed the behavior of the navigation buttons in the browser. The navigation buttons on a browser are from the time of sundials and static web pages. We live in the world of atomic watches and dynamic content. I don’t want you to go back to the last static web page that was loaded. More times than not, that messes up the user’s experience.

I have seen/implemented some solutions. One that we have used is when the first page is served up you start some kind of memory. Start a stack of the user’s actions, saving them in the session. When the user tries to go back to the main page, they are redirected to the real last page they were at. This is good for AJAX intensive applications. These days I am finding the percentage of AJAX requests is many times higher than AJAX requests when I am trying to create a rich user experience. The problem is you get into situations where the user really does want to go back to the last domain they are at, but your application, very annoyingly, will not let them leave. As a user of the internet, I quickly learned that if I hit the back button many times quick enough, the page doesn’t have time to render the redirect, and I can get back. The fact I even know that is annoying, let alone what it takes away from my application.

Another solution is to have lots of static pages. Don’t let the user get too far without loading a new page. This way the back button can really only do minimal damage. Use lots of client side JavaScript and limit the amount of AJAX calls to the bare minimum. This has become industry standard for web pages which involve payment processing. Since the developers can’t guarantee that a user won’t hit the back refresh / or forward buttons, you have to be able to have some kind of state recovery. If the distance between static pages is short, then you don’t do too much damage. However, this constraint limits what you can do with your web application, limiting AJAX calls.

My favorite solution is to take the browsers out back and teach them about feature envy. They are telling me to serve HTML/JavaScript/Flash content inside their main pane architecture, and they will take care of the rest. Well then, keep your hands to yourself!!! Navigating between web applications(domains) makes sense at some high level, because that is in the job of a web browser. They get you from one web application to another. However, inside of my web application, let me decide what I want the back button to do. I understand some kind of backwards compatibility so the Geocities web page with the contra code on it doesn’t get lost (like we don’t all have it memorized). So, let me turn it off for my domain. Give me a JavaScript API to say what the back button behavior should be or let me at least turn it off. There are some attempts to do just this, but they are not straightforward or part of the core library.

So, this will require all developers of web applications to implement navigation that makes sense within their web applications. Great!!! If they are worried at all about user experience, they have already thought about this and have some solution in place. Getting rid of the back button just gives them more power. I have never met a developer who wouldn’t embrace more freedom in their tools.

Software defects are a part of software. This is a negative subject, but I don’t want to seem like the software I write is full of defects and bugs. This blog is addressing how I have seen teams turn a roadblock into a success. I have read and heard at conferences about teams who take small failures and create a culture of failure around it. Following are some examples of small failures I have seen and great successes built around them. Sometimes I see small failures as a path to larger positive culture changes.

I think it is safe to say all software that is used and sufficiently complex has defects. There are many reasons for the defects. Here are a few of the defect situations I have been in and how our team solved them.

I think the important part I learned after evaluating these is that resolution of these situation needs to be fast and high quality. They need to make the project better off at the end of their resolution. Success for the customer is the only true result of a project, and defects may not be the most ideal path, but they must lead there.

The speed is important, but it doesn’t mean I should rush a solution and throw it in as soon as possible. To me it means giving the defect the red carpet treatment. I am trying to capture craftsmanship in the face of adversity. Every project has its ups and its downs, and a craftsman will take pride in success through any obstacles. Instead of making defects a slippery slope downhill, they are one step back and we take two steps forward.

Bugs

Bugs are a part of every piece of software I have written. That statement sounds a lot worse than it is. I have worked on systems where the requirements are dramatically changed week to week (which can be pretty exciting). There are situations I didn’t take into account, or some behavior I didn’t imagine until a real user started hitting the system. Now, as a developer, I do more mental exercises and think thoroughly through my solutions as I get more experience. This has never made my code bug free. For that reason, my team needs to know how to deal with bugs (I am fairly certain I am not the only creator of bugs).

Here is a line from the Pragmatic Programmer book, “It doesn’t matter whether the bug is your fault or someone else’s. It is still your problem.”

So, the team has a bug list. We list out the bugs as a todo list in basecamp, so they are not so formal they can be forgotten about. Then we try to address them and work on new stories. This worked well enough until there was a big release coming up and our customer came back to us with a big list of bugs. We put them on the list and continued to fix them, as well as work on stories. They were getting completed, but there was never an empty bug list. Then, the customer (who can directly add/edit the bug list) started writing priorities to the bugs. This one is HIGH priority. This one is CRITICAL. This one is IMMEDIATE. I make the suggestion “We need some real tool to manage our bug list,” because I can’t fit all this in my head. There isn’t that much room up there and I need to use it wisely. One of my team members suggested maybe it wasn’t the craftsmanship way to have ANY bugs shipped. I was trying to solve the wrong end of the equation. So, the team lead put forth a no bugs policy that we all agreed with.

You can not pick up a new story unless the bug list is empty.

This made sense to me, but I had reservations about a small/insignificant bug taking priority over a story that is important. To date, this has not happened, and the bug list has stayed near no bugs. That doesn’t mean less bugs are found, it just means they are fixed, and the code is refactored to prevent a future occurrence. Most importantly, some tool to track bugs never made it into our system. That was an idea which would have desensitized the team mentality to bugs, whereas with our policy now we are very sensitive to the issue of bugs. Challenging the craftsmanship of the team members that buggy code is something you should take personally was the right choice. Bugs got a first class ticket to termination in our system.

Now, the definition of a bug versus a small feature enhancement is a fine line. I know I have failed to define it well, and that might contribute to what gets called a “bug”. Often times, the urgency in the customer takes up more mental space than me thinking through it, looking up the acceptance criteria for the story where it was implemented, and going back to the customer and saying, “No, that was clearly not a defined scenario, we are going to need a story to turn that button green.” Now the next time, it is more than turning a button green, but the precedent has already been set. All I have been able to do is to strike a balance based upon how much effort it would take to make the bug/feature enhancement work. If it is a lot of effort, I will double check the bug to make sure it is a bug. If it is, I fix it. If not, I will push back to the customer to write a story card.

Production Support

Once a system goes into production, support begins. Following along with one of Paul Graham’s ideas, we have the developers doing the production support. We are the ones who wrote the system and know the system the best. When I look at a production support request, I can not only solve it, but make sure it doesn’t happen again. Or if it does, make sure it is easy to correct.

So, during our first deployment of a system, a single team member stepped up as the “production support” developer. I don’t know if he embraced it or was cornered to it, but as a craftsman, he took the responsibility and ran with it. As further systems were released, he would sometimes be doing an entire day of production support. Production support can be a lot of debugging and fixing data, which can be fun, but more times than not is tedious and rhythmic. Often times when I saw a production support email, I would look to the “production support guy,” who could fix it in about half the time I could. This seems a lot like a silo to me. Everyone should be able to do production support on any system. I should have to, because it is a perspective of the system that is important to have.

In response to this, we came up with a system of triage. Each day of the week is assigned to a specific developer. If a support item comes up, it is the job of the triage developer to respond to the client/customer we are working on it. If it addressed to a specific person, they will inform them. Otherwise it is on the triage developer’s shoulders to fix the support request before they continue their work for the day. This ensures the client always has an open line of communication with a developer. An email never slips through and doesn’t get addressed. There is clear responsibility to who should be addressing the support item. I know the “production support” developer is in favor of this system. As well as the customer, they ask who the triage is for the day and have no qualms about interrupting their work, as they should.

Communication and Managing Expectations

Recently, I did some integration work with a third party vendor. They were developing their side of the integration at the same time as us. Not wanting to slow development and wait for their functionality, we decided to write a mock server and integrate with the host according to the spec from the third party vendor. We received a story from the customer for that and proceeded to make the client for the third party system calls. We finished our story. In the demo portion of the iteration planning meeting, we could only demo against our mock server. This caused some nervousness in the customer (rightly so). I replied, “Once their side of the system is done, we should be able to send our calls across.” Then we received three more similar stories, but different system calls. We did them, removed the duplication and felt really good about the job we did. The came their test server.

Nothing worked! There were all sorts of communication problems, questions about who implemented the system to what spec, and political questions. Despite us thinking we were in the right, the stories were signed and the customer said, “Well, you said this would work.” At first, I tried to communicate the reason why it didn’t work, and how we can move forward. We came to spend a lot of time on this, and the team felt the integration should be its own story. The customer pushed back, “Well, you said this would work.” There was some tension, because both sides were right. We didn’t believe it was our fault it didn’t work (neither did the customer), but we told them it would. Rather than let out the righteous indignation I was feeling, one of the team members mentioned.

“We should not have assured you it would work.”

That one line brought the real problem to the front for both sides. We didn’t know whether it would work or not, just that we wrote the right code to the specification we had at the time. That code by itself has no value to the customer without it working, though. Once that line was said, everyone in the room sat for a second, then understood. The expectation over defect versus behavior was out of sync. A little ownership over the defect was all we needed to ease the tension and move forward. It is unproductive to get stuck in a stalemate of expectations. When in doubt, the customer is always right.

We had a new developer join our project recently, and he needed his computer to be setup with the project. “Here is the svn repository, when you check it out, run these rake tasks.” It unfortunately, is never that easy. This project setup revealed something about Active Record and migrations that I didn’t know about.

When I create a migration, I will often do data manipulation on the database, or pre-populate some fields with data needed for a lookup table. Lets look at a sample migration from a trivia game.

class CreateQuestions < ActiveRecord::Migration

  def self.up

    create_table :questions do |t|
      t.column :text, :string
      t.column :answer, :string
      t.timestamps
    end

    Question.populate
  end

  def self.down
    drop_table :questions
  end
end

I want to add some sample questions, so that even if you don’t have your own questions, you will still be able to play the game. I added the method to the populate model, because I use it elsewhere in the code, and I try to keep it DRY. The populate method on the question model looks like this:

class Question < ActiveRecord::Base
  belongs_to :game
  has_many :answers

  def self.populate
    Questions.create(:name => "What is your favorite color?", :answer => "I don't know")
    Questions.create(:name => "Who was the first President", :answer => "George Washington")
    Questions.create(:name => "Who was born Samuel Clemens?", :answer => "Mark Twain")
  end

end

So, later on, I decided to add a degree of difficulty to the questions, so the players can get more points for answering harder questions. Here is what the migration looked like.

class CreateQuestions < ActiveRecord::Migration

  def self.up
    add_column :questions, :rank, :integer
    Question.destroy_all #In case there are any old ones
    Question.populate
  end

  def self.down
    remove_column :questions, :rank
  end
end

Of which I had to change the populate method on the question class to:

class Question < ActiveRecord::Base
  belongs_to :game
  has_many :answers

  def self.populate
    Questions.create(:name => "What is your favorite color?", :answer => "I don't know", :rank => 1)
    Questions.create(:name => "Who was the first President", :answer => "George Washington", :rank => 3)
    Questions.create(:name => "Who was born Samuel Clemens?", :answer => "Mark Twain", :rank => 8)
  end

end

Then I ran my migrations, and continued development. Then when developer number 2 came across and checked out the project and ran the migrations, he got the error.

undefined method rank= for class Question (…or something very similar)

The problem is the old migration is dependent on the new model. All models in rails are just a mirror of the database, so the new model has a forward definition of the data. The code in the model knows about the rank field, but the schema of the database hasn’t caught up to create that portion of the mirror yet. This creates a little bit of a catch 22. The rails wiki (http://wiki.rubyonrails.org/rails/pages/UsingMigrations) about migrations tells you to redefine the class to stop name conflicts. This would require me to make my migrations model agnostic, inserting straight to the database. As a spoiled brat when it comes to databases and rails, I refuse to let go of my Active Record sugary syntax. Another solution I thought of is to just make the last change to question do the populate, and remove it from the previous versions. This will become a maintenance nightmare.

I came to the realization that I want to make a distinction between form and content when it comes to migrations. Form in this case is schema form, the changes to the database which reflect the data which the Active Records can potential hold. Content is the specific data which is in the database. This distinction allows for me to use the power of my model classes in my data migrations, which is the place it is useful. It maintains backwards compatibility, because before I go touching the data, I have to make sure my schema is right.

What does this look like in Rails? I am not sure yet. Possibly db/migrations/schema and db/migrations/data. Possible saving the data migrations in each migration as a block and executing those at the end, only when you have the schema is correct. I am going to try it out!