Friday, July 9, 2010

Loompa!

There's a nifty little library called Loompa (http://rubygems.org/gems/loompa) that allows one to manage a mess of forks in a fork poolish sort of way. Basically, I would start up Loompa with the number of forks to be run and what method of logging to use
loompa = Loompa.new(fork_num)
After this, I would call the start method and pass it a block, like so,
loompa.start do
  puts "I'm running"
  sleep 10;
end

The code will block and continuously print "I'm running" until I press ^C. The Loompa spawn will print the message, sleep ten seconds, then unregister itself with the parent process and die. The parent process will start up a new child process to fill in the gap left by the vacated spawn.

The Loompa spawn will be listening for the INT, TERM, and HUP signals and will die if it receives any one of them. Along with this, the parent and each child shares two IO pipes to connect them in which they will be listening for each other. If the parent process dies, the children will not have received a "ping" from it and will commit sepuku as soon as their block is done executing.

The library still has a ways to go. If you have a long running block and the parent process dies, the children will continue to execute. If you have something like Monit keeping processes alive, it will recognize that the parent process is gone and will start a completely new stack. You could easily run into a situation where your memory disappears and the CPU load goes into the 400s. The best thing to do is make sure your processes don't run for more than, say, 10 minutes. The situation still exists but it may be minimized by that.

You can also start Loompa with a logging module/class
loompa = Loompa.new(fork_num, Logger)
It expects the logger to have at least three methods, info, debug, and error. Here's the default logger included in Loompa:
module DefaultLogger
  def DefaultLogger.info(msg)
    STDOUT.puts msg
  end
  
  def DefaultLogger.debug(msg)
    STDOUT.puts msg
  end
  
  def DefaultLogger.error(msg)
    STDOUT.puts msg
  end
end

Good luck :)

No comments:

Post a Comment