This is what happens when you blog drunk

Meezy’s incoherent rants ftw.

Rails/Web App Performance, Part 1 – Measurement

A performance writeup series holiday present to everyone. This is the first of a 4 part series:

1. Measurement
2. Backend Optimizations
3. Server Side Tweaks
4. Frontend Optimizations

Numbers, not Feelings

Before I started reading about performance, but was tasked with optimizing something, I’d go with whether or not it “felt” fast. While this is a somewhat acceptable way to determine if something needs to be optimized, it’s not a good measure of optimization.

After putting hard work into some optimization, you’re going to want to see improvement. So much, so, that if left only to your own senses, odds are you’re going to see improvement. Even if you’ve made things worse. Because of this, it’s important to go by benchmarking and profiling tools, and not feelings.

Another reason it’s important to rely on tools and not feelings is that they allow you to hone in on what’s actually slow. There’s a lot of things that go on in a web request, an example being you send a request to Apache, which forwards it to Mongrel, which spins up some Ruby action, which then pipes it back to your box. You might see something in your backend code and say, “I know that’s slow, I’m going to speed it up.” Unfortunately, without a baseline measurement, you (1) don’t know how much the improvement will help, and (2) you can’t be sure that it needs improvement.

Numbers justify everything. Without them, it’s hard to explain to others what exactly you’ve been doing for a week.

Statistics 101

I was lucky enough to take a statistics class in college. Lucky in that it was an easy A. Unfortunately, I don’t remember much else about it. I’m assuming you’re in about the same position.

Though I think it’s taught as an axiom or something about sample size, in casual conversation I’ve heard it referenced as the “law of small numbers.” Essentially, if you don’t have enough samples, you can’t be entirely sure what you’re measuring. You could be measuring the time of Ruby’s garbage collector, when you really want to see how long a regex match is taking. This wouldn’t only lead to inaccurate results, but it might misguide your optimization efforts. In short, running a test more times is better.

While taking statistical measurements, it’s important to reduce confounding factors. Anything that could interfere with the numbers you’re gathering can skew your data. If you’re running a benchmark locally, close down other applications. Again, it’s hard to know if you’re actually measuring Ruby’s regex matching speeds if you’ve got Lord of the Rings playing on iTunes and are playing Tower Defense on Firefox. Maybe placing that water tower just hogged some CPU time, making your Ruby slow down. The timer’s won’t know, and neither will you.

If you’re testing server throughput, be sure that you’re testing as close as possible to the machine. If you have a sibling server that’s 5 feet from it, that’s better, because you’re not measuring other router speeds or black hole internet spots.

Lastly, when presenting your measurements, calculate a standard deviation along with the mean. This is incredibly important. A standard deviation indicates how far measurements deviate from the mean. One standard deviation will cover something like 75% of the points, and a second standard deviation will I think be 92% of the measurements. Though there’s no built in Ruby standard deviation calculation, I’ve provided one below.

If you have a request that shows it’s only taking half a second on average, you can think, “this is great, our application is so fast!” But if you couple that with the related standard deviation, and it’s 12 seconds, you know some people are waiting a hell of a lot longer than half a second. This could reveal something like a mongrel hanging or a race condition that just a mean wouldn’t provide.

Toolbelt

There’s lots of tools out there to keep up with how well and healthy and quickly applications are running. It’s good to get familiar with their strengths and weaknesses, and see what works for you. Also, I’m only covering free tools here, because nobody who offers a paid tool is paying me to post it, and because these tools have lots of developer contributions and use out in the wild.

As I go through this series, I’ll demonstrate using each tool in depth, but for now, here’s some things you can just read about.

For the front end, there’s a fantastic tool called YSlow that’s an addon for Firebug. It’s not a measurement tool as much as it is an analysis of what your browser has to do before it can display the page. The analysis offers several suggestions and explanations of how you can both reduce load on your server and speed up rendering time in your browser.

For Ruby, if you have a question about how fast an operation is, there’s a built in benchmark library. It’s not the most robust tool, but for simple questions, it provides quick and reliable answers.

For Rails, there’s RubyProf. I know RubyProf is technically used to benchmark Ruby, but since this is webapps we’re talking about, and RubyProf comes with a built in Rails plugin, it’s my favorite tool for the job. Also, for questions regarding production performance application, there are log analyzers. I’ve written one, logpwnr, which works out of the box on OSX. There’s also a tool called Rails Analyer that does the same thing but is much prettier. It doesn’t work out of the box on OSX. I’ll try to cover it on the backend post, though.

To test server throughput, the standard tool for the job is httperf. It’s got an arcane interface, but does a great job at slamming servers to show peak potential throughput.

Something to think about when you’re using these tools is whether or not what you’re measuring needs a priming. If it’s a webpage, do you want to measure things with or without your browser cache? If it’s a Rails app, it’s important to let everything spin up a few times to ensure things are cached. The first time a Rails app is hit is usually incredibly slow, and is such an edge case that it’s not worth optimizing, for example.

Next up, I’ll talk about backend performance.

Resources

• Zed Shaw’s Statistics Rant
• Standard Deviation Code Snippet

Thank your arm, Bradford

I watched part of the Heisman Trophy Presentation last night. I thought Tebow should have won, with Bradford coming in second (McCoy is responsible for Texas’s loss – see: pick6). But Bradford gets up there and starts his acceptance speech saying he wants to “thank God, that’s how I’m able to be up here” or something like that.

He cheapened the award with that by appealing to superstition. Did a god really pick you, Bradford, to win the Heisman? It’s not a result of a rigid practice regiment, studying, and genetic gifts? Does that mean he picked the guy on the side of the road to be homeless? It’s all part of a plan?

Superstition is good at justifying events after they happen, but rarely predicts events accurately. That’s how you know you’re dealing with superstition and not reality. I can drop a glass cup from a 4 story building and predict that it will break, explaining how the force that will stop the cup from its speed will be greater than the structure of the glass can withstand. There’s billions of similar predictions that can be made.

If you pray for something, or bring a rabbit’s foot, it has no bearing on predictions, and doesn’t even provide one. Pray for a good test grade, if it happens, is it prayer or studying that helped? If it doesn’t happen, is it not part of the superstition’s plan?

And that, there, is the absurdity of it all. Take responsibility for what you do. Bradford, you played outstandingly during the season, put up prolific numbers, and have led your team to the BCS title game. That was you, not your rabbit’s foot or your Sunday morning rituals. Homeless guy, it wasn’t the devil that made your bad choices, and I’ll be happy to help you out, but admit that it’s your own actions that got you where you are.

Baby Monitor++

Since we got pregnant with Nate, Sara had pushed a video baby monitor. She said it would be helpful in knowing what the kid was doing (obviously selling point). I acknowledged that it would be helpful, but said it wasn’t really important to see what they’re doing, you just have to hear it. Also, having a camera in our house is creepy, largely for the reason that you can’t be sure who’s watching.

I looked at video baby monitors, and found none that were encrypted. They broadcast different data over several frequencies, making it harder for someone to just pick up and interpret, but there’s nothing stopping someone from buying the same monitor and driving around a neighborhood to see who he can spy on.

This bothers me for a few reasons. First, I don’t want some random guy looking at Nate when I’m changing his diaper. Second, I’ve got a pretty reasonable expectation to privacy in my house, so if I want to walk around without pants or whatever, I don’t feel like broadcasting that (though Sara does yell at me to close the curtains in the morning, and I just argue that the neighbors get a free show).

Well, for Christmas (we rarely wait til the actual day to exchange gifts), I got Sara a video monitor. Kinda. I got her a Panasonic Network Camera. The network camera works a lot differently from a regular baby monitor. First, the one I bought was wired, so it’s only $99 (yay for cheap!)

Next, since the camera is wired, the broadcast from the camera itself can be entirely closed-circuit. It’s not, as it hooks up to a switch in the attic which hooks up to a wireless router, but the wifi is encrypted enough to broadcast credit card data, so I feel safe with it.

So the camera hosts its own webserver, and you can just head to its IP address on any browser within our network, and you get to control a camera in Nate’s room. Already a win, but I decided to make it super friendly for Sara.

I changed all the /etc/hosts files in our house to have an entry for “nate.” Now, on any computer in the house, you can open a browser and just type “nate” in the url bar, and you get to see the camera.

But wait, it gets better. Safari has a feature called Web Clips. Web Clips allows you to save a section of a web page, or, say, an auto-updating image, to a widget. On OSX, you can just hit F12 (by default) and several widgets come up. So Sara can just press F12 on any computer in the house and boom, she’s looking into Nate’s room.

This wasn’t a one step solution like a baby video monitor sold at Babys’R'Us, but hooking up all of the different hacks together provided a much more secure and robust solution. Plus I put a switch up in the attic, in case I need to wire anything else up. Essentially, this setup is cooler than yours, and I wanted to point that out.

Backfired

Nate loves our remote control, but I don’t like him playing with it because he broke the previous one, and, well, they’re expensive. He just had it in his hands and had turned off the Cowboys-Steelers game.

I took the remote from him, asking, “Do you want to watch Bunny Town?” After securing the remote, I said, “too bad. You can watch after the game.” To which he responded by throwing a fit, and Sara made me put on Bunny Town.

Damnit.

That you, Santa Claus?

Reading the bajillion skeptic/parenting/atheist blogs I do, and with Christmas (that’s right I said Christmas, not that it really matters as long as there’s a tree and presents for the kids) time coming up, I’ve been happy to see several posts addressing Santa Claus (side note: I’m assuming it’s spelled sans ‘e’).

Sara and I had different experiences with Santa, from what we remember. Sara remembers getting big presents from Santa, and small gifts from her parents. I remember receiving a mix. Neither of us placed cookies by the tree.

I worry about going through all the motions and convincing Nate there’s an omniscient fat man living up north who can do magic, but I’m hoping that it’s just a young enough thing where, when he does find out there’s no such thing, there’s no harm done.

Sara persuaded me to go through with the Santa thing because (1) it would be weird to have a 4 year old (thinking for the future) who crushed every other 4 year old’s dreams on the subject, (2) he might feel left out from other kids who do receive gifts in such a way, and (3), it’s a lot of fun to pretend there’s someone who knows and cares what you’re doing all the time (for adults, this is called religion, and requires no mall-lap-pictures).

So Nate’s going to receive a mix of presents from us and Santa, and we’re definitely going to do the cookies, because kids like baking anyways, and FREE COOKIES.

And a side question, is the Easter Bunny even talked about anymore (not counting South Park)?