Cap Release Notes

One of the really good things I learned from some guy, we’ll call Donkey, is to have everyone write down deployment dependencies in a file, so if someone goes on vacation or whatever during a deploy, you at least have an idea of what you should do.

I put it in /README, and separate everything by sprints. It looks something like this:

Even though everyone tries to remember to put things in there, there’s still a single point of failure, when the idiot who deploys it forgets to check the readme.

So, to prevent future self-indictments, I put a task that runs at the end of our cap script that prints the current release notes.

Ideally all of these are handled with a comprehensive deploy script, but that’s not always feasible/doable. Here’s the cap task, and if you make it the last thing that runs, you’ll get a friendly reminder at the end of every deploy for things you need to do.

Quick Tips

Chrome view source
I ran into an incredibly confusing bug last week where I had some fields named user[field_x], but they were being posted to my Rails server as user[field_y]. I started to blame all sorts of crazy things (becoming superstitious in not knowing wtf could be happening).

I pulled my field-level identity map (another blog post). I wrote some Rack middleware to intercept the parameters before Rails’s ParamsParser could touch it and make sure they were good. Fail. Nothing worked. After 2 hours of fighting with it, unable to reproduce in tests or anything, I realized I wasn’t viewing the source of the content I was actually viewing.

Chrome sends another, new request to the server when you say view source. This means that you’re not viewing the source from the page you’re viewing. If the pages are static or if there’s no way server state affects your page, then you’re good. Otherwise you might lose 3 hours.

This bug was painful enough that I’ve dropped Chrome as my default browser and am back on Firefox. I hope that by reading this you’ll think of it when you do view source on Chrome and nothing makes sense.

Git stash
I learned about git-stash over the weekend. It allows you to stash your changes on a stack, make some other changes to the codebase, do what you want with them, and then pull those stashed changes back out.

Example use case: I branch before everything I work on, but eventually I have to merge stuff back in. So I branch to make feature X, get it written, then merge it back into HEAD. Unfortunately, tests are failing. I’m not going to push my changes yet, but something needs to be changed for another developer who wants something to run locally. git stash –keep-index; do changes; git push; git stash pop. Done and done, and no broken builds!

Timecop
A developer at our office was working with some relative time methods. These are inherently tricky to test, because if “next week” means “next business week”, you can’t just willy-nilly add 7 days to a day and do tests. Same with “this week”, etc.

Instead of doing a bunch of complicated date math that would arguably make the tests just as scary as the code, we found Timecop, which lets you freeze time for your tests. You can say, “I want Date.today to evaluate to 20Apr2010″, and it will. This allowed for significantly simpler tests and saved a ton of time.

Avoid toggle actions
Even on some of the more popular sites on the internets, there are a lot of AJAX actions that don’t handle network failures well. Backpack (at least on my iPhone) just keeps the in progress icon going forever. We’re not doing anything super-critical with AJAX on our app, but we’d like to at least handle timeouts or failures gracefully.

The simple solution is to check for the HTTP response code and act appropriately (2xx? Yay! 4xx? BOO!). We decided to do this, but realized that it’s possible that your request makes it through to the server, then your network fails, and you get a timeout. What do we say happened on the client side?

If they’re single-direction actions (e.g., marking something done), then you can just say “Something went wrong,” and not worry about changing it back or anything. If it’s a toggle, though, you can’t make any guarantees. “Something went wrong,” sure, but should the user try again? What if it only went through for the first 3 items?

As such, we’re no longer writing toggle_x actions. Now we’ll handle that on the frontend, and have negative_x and positive_x methods.

DelayedJob scheduling improvements

Last week I talked with a guy who noted that the scheduling patch I’d blogged about for DJ was insufficient for scheduling something every day at 8AM. This is because running a job and then scheduling the job again after a success pushes the job out a little bit each time (maybe in a week or so it runs at 10AM, e.g.)

I thought about trying to make a run_at class method that would let you specify it, like “run_at ’8am’”, but that quickly fell through when I realized ’8am’ is ambiguous – 8AM every day? Every Wednesday? Every 4th week in July?

I fall inline with Brandon Keepers when I say that I don’t want to re-implement cron, because (imo) cron is one of the best scheduling tools out there already, and it comes packed onto every Unix distribution that ends with x.

Instead, I changed the run_every method to now accept a block, so you can pick the new best time to run if you’d like, or just keep using the same old (8.hours) syntax.

Think something like this:

If you like it, here’s the GitHub Repo.

Machinist + RSpec matcher

Most of the time when I’m putting the skeleton of my Rails app together, I end up with a test like this:

After discovering Remarkable this week, I decided it was worth putting that into a macro matcher*. So now you can just say it { should work_with_machinist }. Here’s the code. Not tested. Enjoy:

* Thanks @David for pointing out it’s a matcher, not a macro

** Clarification: From reading the comments, I realize that this doesn’t seem useful unless you’ve used machinist. Machinist’s make method works on an ActiveRecord model, and works something like this: calls .new() with the fake params, calls save!, and then calls reload. If any of those fail, it raises an exception, so this effectively tests your blueprints for validation and callback sanity.

Bling Bling gem management

With Rails 3 and 2.3.5, I’ve been trying to use Bundler, but it’s been pretty painful so far. It’s not lightweight, and the only problem I really had with config.gem was that it had to load up the Rails environment to run, which meant if you had some code that was out in the open referencing a module or class in a gem that wasn’t yet installed, it borked (AASM did this to us). This made rake gems:install useless for lots of things.

After jumping through the bajillion hoops and kludges to get mongomapper working, RSpec started giving me some problems with bundler, so I set out last night to make something really simple, just to manage gems, Bling.

It’s not nearly as feature-rich as Bundler. It won’t unpack your gems or cache them, and it wants to install all gems to your system. So if those features are important to you (and they probably are to a lot of people), bling isn’t for you.

Bling just uses a yaml file (in Rails.root, for Rails projects) to keep up with gems, and lets you specify gems, their versions, their sources, and what lib to require (if not easily guessed as the gem name).

I bet I’ll have to add features to it as I go, but it’s solved the few problems I’ve had so far.

To get started, just do a “(sudo) gem install bling” and then in your Rails directory to a “bling init”. Once you’ve set up your bling.yml how you want, if you have any new gems, just do a “bling install”.

Gem info is here. GitHub is here.

Scheduling Jobs with DelayedJob

For a new project at work, I was tasked with reevaluated our async worker system. Previously we’d been using beanstalkd with a jobs model that used AASM for persistence and kept up with when something should be scheduled (and was scheduled with a rake task). After looking at the different things available, I decided DelayedJob was a better way to go, because it involved fewer dependencies (no more cron-rake work, no more beanstalkd) and had the persistence built in, instead of our custom code.

One shortcoming, though, seemed to be scheduling jobs. After thinking about different ways of keeping up with what jobs could be scheduled, I figured it’d make the most sense for jobs to just re-schedule themselves before execution.

That said, I didn’t want to put that in every class that had a perform() method as a boilerplate, “reschedule myself for 24 hours,” because it’s harder to maintain and grok what jobs are scheduled, and because it’s not nearly as readable.

Enter ScheduledJob, a mixin that gives you a run_every method on your class:

With this, you can create a class and include ScheduledJob, and then just say “run_every 24.hours” or whatever. Like this:

Hopefully this is helpful. I’ll need to write some tests, and see if the DJ guys want to use it on GitHub.

Secure Passwords sans SSL

I use a lot of webapps that don’t use SSL. I don’t think this is necessarily bad – SSL costs money, is slower, introduces development and IT headaches – but I do worry that I’m sending my password in cleartext.

I set out tonight to offer a solution for that. I haven’t tried to work it into any existing plugin, but just the links and the code should suffice for someone who doesn’t want to let passwords go readable across the wire.

I decided that I would send an HMAC using the SHA-256 encrypted password as the key, and the username and a unix style timestamp concatenated as the data. To make this work I dropped password salts. I haven’t decided if it’s safer to send the salt to the user or just to drop it. Hints in this area would be appreciated.

Prerequisites for this are jQuery and the ridiculously-easy jQuery sha256 plugin.

To start things out, I created a Rails app (using 2.3.4, if you’re curious) with a User model that had a login and password field. I meant for the login to be, well, the login, and the password to be a SHA-256 hexdigest of the actual user’s password. If you adapt this code to work with RestfulAuthentication, which handles the password digests nicely, you’ll have to solve the salt problem mentioned above either by dropping it or sending it to the client.

Here’s how I set up my user:

Then, I needed a login form. Here’s the relevant HTML:

As you can see, as soon as a user submits the form, I hash the password and create the digest, and then blank the password field out. Obviously that last part is important, because otherwise you’re sending the user’s password as cleartext.

This code is more of a jumping off point than a landing place, but here’s what my (trivial) action to login looks like:

The meat and potatoes of the authentication goes in the User model, and the code looks painfully similar to its Javascript counterpart:

Essentially this goes, “ok, if you took a hash of your password locally and then used that to hash whatever timestamp you received and your username, and I get the same result on my end, it’s really likely that you’re the right user.”

Now, this is almost secure, but not quite. There’s still nothing preventing someone from re-posting the same data and logging in. This is just as bad as simply sending the digested password over to the server. I suggest (if you’re not using SSL), sending neither the password or the digest of the password itself over the wire.

The benefit of using the timestamp is that you can let the client send 1-time valid digests. So, I added a new model to my Rails app, LoginRequest, consisting of a timestamp attribute, and a User has many of them.

Here’s what the more secure version of authenticate looks like:

Now, even if a bad guy snoops everything you sent to the server, he (or she!) can’t reuse it, because that timestamp’s been used. Changing the timestamp should significantly change the hash, and without the secret information necessary for the encryption (the password hash), it’s not feasible to guess.

Also, brute forcing against a single timestamp isn’t an option, because even *failed* login attempts are registered as a LoginRequest. You get one try per timestamp to login.

If you wanted to, you could even put a check in the authenticate method that required that the timestamp be from the last day or some other arbitrary time limit.

So, if you aren’t storing important data, and for some reason can’t use OpenID, this seems to be a candidate for at least keeping passwords from going out in the open. I’m pretty sure there’s some hole in it that I haven’t seen, so if you see something, please put it in the comments so I might fix it. Thanks!

* Note: I used OpenSSL’s HMAC functionality over ruby-hmac because (1) the author of ruby-hmac suggests as much and (2) Nathaniel Bibler showed significantly better performance from the OpenSSL implementation here: http://blog.nathanielbibler.com/post/63031273/openssl-hmac-vs-ruby-hmac-benchmarks

Async Observer sans Rails

My favorite background job pairing is beanstalkd with async_observer. I think the code that comes out is clean, easily pushed out to a queue, and beanstalkd is stupid easy to get going.

Both at work and on a side project, I’ve run into a situation where I need to put something into beanstalkd to be processed by a Rails-backed worker, but don’t want to load up the Rails framework to do that. Easy example: receiving email, I’d like to queue the processing how Jason Seifer suggests, but don’t want to have my own beanstalkd tube dedicated to email processing.

When you do an async_send, it just gets serialized as a string out to beanstalkd, with something like Model.find(id).method()

So if you want to push stuff into your queue in a rake task without having to load up the Rails environment, just use this method:

At work, we have a job system, and cron runs every 2 minutes to find scheduled jobs that should run and put them into the queue. Initially this was coded as a Rails-backed rake task, which meant every 2 minutes you’d see a ruby process load up for 10-15 seconds and steal 95% of a CPU. As we had more applications on the same machine needing jobs loaded, this wasn’t effective. Making the task only use beanstalk-client (and mysql), the task now runs in <1 second and doesn’t even register on top.

Bgtwt Open Source’d!

After (I think) removing the passwords and everything else sensitive in the code, I’ve open sourced the code behind bgtwt.com.

I’m not sure if it will get any patch submissions or not, but I figured with it being closed, it definitely wouldn’t. If you submit anything you’re proud of (you can find the code here: http://github.com/terrbear/bgtwt), and I like it, I’ll happily put it up on the site.

You’re also free to copy the code and do whatever you want with it. I’ll attach the proper license in a few days.

If you’re looking for a feature that would be awesome and want to start playing with Rails, I’d like to integrate acts_as_unvlogable into bgtwt.

Rails and OpenID: Start Your Engines

(Apologies for the ridiculously lame title, but I laughed while writing it, so it stays.)

If you care about your user’s security, and your time, it’s time you stop asking them for passwords. Storing user passwords and handling authentication offers a bunch of problems that have already been solved, and requires a lot of work to do it right.

Think about it. If you’re storing user’s passwords in your database, you’ve automatically got sensitive information on your server. Maybe it’s just a College Football Pick ‘Em site, which stores nothing too important … until you store a user’s password. Let’s face it, most users don’t use a new password for every site, so if someone breaks into your server, you’re responsible if that information is lost.

So, you’ve got to make sure your server’s security is up to date. Then, it’s still possible there’s a break in, or you have a malicious developer. So you have to encrypt the passwords. But not just encrypt them, because that would still be susceptible to dictionary attacks. You have to salt each password.

What about brute force login attempts on your sign in page? Have you handled that?

These are just some of the problems that you have to solve the minute you decide to store someone’s password.

A better solution? OpenID. Below I’ll explain how to use my newly written (really repackaged) OpenID Rails engine, and you can be on your way to making the world an easier place to sign in.

Read the rest of this entry »