Async Observer sans Rails

My favorite background job pairing is beanstalkd with async_observer. I think the code that comes out is clean, easily pushed out to a queue, and beanstalkd is stupid easy to get going.

Both at work and on a side project, I’ve run into a situation where I need to put something into beanstalkd to be processed by a Rails-backed worker, but don’t want to load up the Rails framework to do that. Easy example: receiving email, I’d like to queue the processing how Jason Seifer suggests, but don’t want to have my own beanstalkd tube dedicated to email processing.

When you do an async_send, it just gets serialized as a string out to beanstalkd, with something like Model.find(id).method()

So if you want to push stuff into your queue in a rake task without having to load up the Rails environment, just use this method:

At work, we have a job system, and cron runs every 2 minutes to find scheduled jobs that should run and put them into the queue. Initially this was coded as a Rails-backed rake task, which meant every 2 minutes you’d see a ruby process load up for 10-15 seconds and steal 95% of a CPU. As we had more applications on the same machine needing jobs loaded, this wasn’t effective. Making the task only use beanstalk-client (and mysql), the task now runs in <1 second and doesn’t even register on top.