8th Light Blog

We had a new developer join our project recently, and he needed his computer to be setup with the project. “Here is the svn repository, when you check it out, run these rake tasks.” It unfortunately, is never that easy. This project setup revealed something about Active Record and migrations that I didn’t know about.

When I create a migration, I will often do data manipulation on the database, or pre-populate some fields with data needed for a lookup table. Lets look at a sample migration from a trivia game.

class CreateQuestions < ActiveRecord::Migration

  def self.up

    create_table :questions do |t|
      t.column :text, :string
      t.column :answer, :string
      t.timestamps
    end

    Question.populate
  end

  def self.down
    drop_table :questions
  end
end

I want to add some sample questions, so that even if you don’t have your own questions, you will still be able to play the game. I added the method to the populate model, because I use it elsewhere in the code, and I try to keep it DRY. The populate method on the question model looks like this:

class Question < ActiveRecord::Base
  belongs_to :game
  has_many :answers

  def self.populate
    Questions.create(:name => "What is your favorite color?", :answer => "I don't know")
    Questions.create(:name => "Who was the first President", :answer => "George Washington")
    Questions.create(:name => "Who was born Samuel Clemens?", :answer => "Mark Twain")
  end

end

So, later on, I decided to add a degree of difficulty to the questions, so the players can get more points for answering harder questions. Here is what the migration looked like.

class CreateQuestions < ActiveRecord::Migration

  def self.up
    add_column :questions, :rank, :integer
    Question.destroy_all #In case there are any old ones
    Question.populate
  end

  def self.down
    remove_column :questions, :rank
  end
end

Of which I had to change the populate method on the question class to:

class Question < ActiveRecord::Base
  belongs_to :game
  has_many :answers

  def self.populate
    Questions.create(:name => "What is your favorite color?", :answer => "I don't know", :rank => 1)
    Questions.create(:name => "Who was the first President", :answer => "George Washington", :rank => 3)
    Questions.create(:name => "Who was born Samuel Clemens?", :answer => "Mark Twain", :rank => 8)
  end

end

Then I ran my migrations, and continued development. Then when developer number 2 came across and checked out the project and ran the migrations, he got the error.

undefined method rank= for class Question (…or something very similar)

The problem is the old migration is dependent on the new model. All models in rails are just a mirror of the database, so the new model has a forward definition of the data. The code in the model knows about the rank field, but the schema of the database hasn’t caught up to create that portion of the mirror yet. This creates a little bit of a catch 22. The rails wiki (http://wiki.rubyonrails.org/rails/pages/UsingMigrations) about migrations tells you to redefine the class to stop name conflicts. This would require me to make my migrations model agnostic, inserting straight to the database. As a spoiled brat when it comes to databases and rails, I refuse to let go of my Active Record sugary syntax. Another solution I thought of is to just make the last change to question do the populate, and remove it from the previous versions. This will become a maintenance nightmare.

I came to the realization that I want to make a distinction between form and content when it comes to migrations. Form in this case is schema form, the changes to the database which reflect the data which the Active Records can potential hold. Content is the specific data which is in the database. This distinction allows for me to use the power of my model classes in my data migrations, which is the place it is useful. It maintains backwards compatibility, because before I go touching the data, I have to make sure my schema is right.

What does this look like in Rails? I am not sure yet. Possibly db/migrations/schema and db/migrations/data. Possible saving the data migrations in each migration as a block and executing those at the end, only when you have the schema is correct. I am going to try it out!

When I first started writing code in ruby, it was a breath of fresh air after writing C# code for a year. Ruby had a thesis, a clear purpose, rather than a hodgepodge of features strung together. It was a language obviously written by someone who cared about what the code looked like. So I jumped into it and loved writing ruby code (still do!). Recently I had to write a project in Java using IntelliJ, and the powerful IDE was also a breath of fresh air after using text editors to write ruby code. The IDE helps me in the same way ruby helped me. I write less code that does the same thing, without loosing expressiveness or transparency of intent. Less code in this case meaning less keystrokes.

So, I am going to do a little experiment using two editors and languages: ruby with textmate and java with IntelliJ. I want to see how many lines of code I have to write. The application I am going to start is a simple baseball at bat scorer. First lets see the ruby version.

We have about 27 lines of code written, and other than the describe method, which I have a macro for, I typed them all by hand. Lets look at the java code. This is just the JUnit test I wrote, there is no production code yet.

It is about 16 lines of code, which I wrote all of but the import statement. By doing a few alt-enters on the squiggly lines, I can get a stub of a class looking like this.

That is 13 more lines of code, without really typing, just hitting enter a few times. Now, lets start to make the method pass. I am going to write the algorithm without worrying about all the type definition java wants. Here is what it looks like.

So the code written is around 4 lines of code. Lets use the IntelliJ autocomplete features to help us make a passing test, which looks like.

So roughly, the ruby version was 23 lines of written code and the java version was about 20 lines of written code. There is an expense to doing all of the alt-enters to auto-generate the method and variable stubs, but IntelliJ is pretty smart about what it generates. Also, I understand a single test doesn’t tell the whole story, but it is a good indication. I found as I wrote more tests for this application, a smaller amount of keystrokes was needed for the java version, and the same amount was needed for ruby. This is due to the refactoring tools, and intellisense. As applications get larger, I find the IntelliJ refactorings become more useful. Inversely, ruby refactorings become more painful, as they are mostly done by hand.

So, in the end the constraints on a static language allows the IDE to make the refactoring tools better. Specifically, when writing java code, I can lean on the IDE to generate all the uninteresting stubs for me. All I do is fill in the algorithms: the fun part. When I start to see places where my code can be cleaned up a bit, I can lean on the IDE again to do those refactorings for me once I recognize the need for them. Removing and optimizing the code is something which is easily deducible in static languages, as it is mostly pattern matching, with no monkey patches, meta-programming, and evals to worry about. This is a limitation which frustrates me as a developer who wants to have a large set of tools in my bag, but is helpful when it comes to developing a powerful IDE.

The number of lines of code I need to type isn’t the reason I choose a language over another. In fact, it would be pretty low on the list of deciding factors. However, it is interesting to see what each language and its sets of tools do best. Hopefully the ruby community can take some notes from them. I would rather solve the other end of the equation, get a powerful ruby IDE. I know Eclipse and Net Beans have some preliminary refactoring tools, but they are still aways from being as seamless as their java counterparts.

As we work with our clients to delivery new features to them quickly and with high quality code, we are finding that the fastest way forward is often writing code in Ruby. The problem is that in many cases we are working with an existing application written in a different language. We may need to add a feature to a C++ application or integrate with a Java Library. One of our favorite solutions in this situations is to have multiple applications talking to each other over sockets.

We’ve found that using sockets to communicate between applications is a great way to ‘bolt on’ new functionality. This type of communication has been around for a long time and nearly every language has socket communication in standard libraries or in free open source projects.

Dean Wampler suggested that Applications Should Use Several Languages and socket communication is another way to enable this.

Socket Basics
Most internet communication is built on socket communication, so I won’t try to go into great detail here. I don’t pretend to be an expert. Here is what you need to understand about sockets to use them in your applications.

UDP vs TCP
There are two basic types of IP socket protocols: UDP and TCP. UPD is what I’ll call a ‘send and pray’ protocol. In UDP there is really no difference between server and client. Each node can listen on some port and write on some port. Messages are sent from point to point or broadcast to all nodes. Either way, delivery is not guaranteed. The message will likely make it to the destination, but there are a host of things that could happen that would prevent delivery. Worse, as the sender you can’t know whether of not your message made it unless you implement some kind of response protocol on top.

I’ve used UDP communications for monitoring / metering applications where it isn’t important that every message be delivered, but I’ve also found that I usually end up implementing some kind of delivery guarantee mechanism anyway.

TCP sockets build another layer on top of basic IP communication that does guarantee delivery. Once the connection is established between client and server, both side can communicate knowing that their message will be received. On the Server side, a socket listens on a particular port for new connections. Most libraries call this operation ‘accept.’ A client ‘opens’ a connection to the server. This process involves the client and the server agreeing on a new port for communication. If the server is going to accept multiple connections, then it usually launches a new thread to handle each new connection.

I use TCP Sockets for most inter-application communication. It turns out that in most situations, I want that guaranteed delivery and don’t want to bother with my own acknowledgment protocol. So it’s the TCP Socket that I’ll show you for now.

Over my next few posts, I’ll show you how to communicate via sockets in many languages. We’ll start with Ruby. Ruby has TCP Sockets built in, though there really isn’t much documentation on how to use it. Here is how to do it.

The TCPSocket class can be found in ‘socket’

require 'socket' 

Create a socket server. The first parameter is the address to listen on. If you specify ‘0.0.0.0’ then the server will accept connections from any host. If you specify ‘localhost’ or ‘127.0.0.1’ this the server will only accept connections from sockets on the same computers. The port can be anything you want. Be sure to avoid commonly used ports.

   server = TCPServer.new('localhost', '4138') 

Now, when you call accept on the server it will block until a client opens up a connection.

    server_socket = server.accept

Now that the server is waiting for a new connection, let’s open that connection from a client. Note that this needs to be happening in a separate ruby process. The first parameter is the host address or name of the server. The second is the port on which the server is listening.

    require 'socket'
    client_socket = TCPSocket.new('localhost', '4138')

Now that there is as new connection open the client and server can talk back and forth at will. In this example, the server just echos the clients message back to it. So all together, the Server side looks like this:

    require 'socket' 
    server = TCPServer.new('localhost', '4138') 
    server_socket = server.accept
    message = server_socket.recvfrom(256)[0]
    server_socket.write("You Said: #{message}")

And the Client side:

    require 'socket'
    client_socket = TCPSocket.new('localhost', '4138')
    print "Enter Message:  "
    input = gets
    client_socket.write(input)
    response = client_socket.recvfrom(256)[0]
    puts response

So that’s it, you are communicating between two applications with sockets. If we want the server to continue processing messages, we can loop reading from the socket:

    require 'socket' 
    server = TCPServer.new('localhost', '4138') 
    server_socket = server.accept

    message = ""
    until message.chomp == "quit" do
        message = server_socket.recvfrom(256)[0]
        server_socket.write("You Said: #{message}")
    end

So far our server can only handle one connection at a time. Often times, you’ll want multiple client to be able to open up connections to the server. To do this, we’ll need to launch a new thread each time a client connects. This is a little class that does that. You can implement your own handler object to handle each socket. The handler only needs a ‘serve’ method that takes a socket. The ThreadGroup is a nice little class to keep track of the threads. When a thread finishes, it is automatically removed from the thread group.

require 'socket' 

class MultiTcpServer

  def initialize(handler, port)
    @group = ThreadGroup.new
    @handler = handler
    @server = TCPServer.new('0.0.0.0', port) 
  end

  def connections_open
    return @group.list.size
  end

  def run
    begin
      loop do
        socket = @server.accept
        @group.add(Thread.new {@handler.serve(socket)})
      end
    rescue IOError
    end
  end

  def stop
    @server.close
  end

end

Hopefully this little tutorial was helpful. Next time I’ll take a look at sockets in C++ or Java.