passenger - Rails app randomly crashes with error "Premature end of script headers"

Question

I am hosting a Ruby on Rails 2.0.2 application on DreamHost. It is on an Apache 2 server, running on top of Phusion Passenger.

The application often returns a 500 error "Rails application failed to start properly", but at random times. It appears to happen when the application is under higher load, though I can't confirm this. It only gets about 2,000 pageviews per day, so I don't think load should really be an issue.

The Apache logs correlate these 500 responses with the error: "Premature end of script headers". Just looking at the logs at 9 AM today, the error appears as often as three or four times a minute. This is clearly unacceptable.

Less frequently, the application hurls a Phusion Passenger page with a stack trace and the error "Broken pipe".

The Rails logs do not list any of these errors.

This happens on both the staging and live sites hosted on DreamHost, but I can't replicate it on a local development server.

So I guess the real question is: Where do I begin to debug this problem?

Answer 1

It turns out that I was merely hitting my memory cap on the DreamHost shared server.

I was running several Rails apps under one account, many of them just for testing and prototyping. Rails uses a lot of memory and so I was quickly reaching my allocation. Support told me that "I checked our logs and it's killed one of your ruby processes 2325 times over the last three days". Whoops.

The solution: try not to run Rails in a shared environment, if you can help it. I'm going to switch at least one of my apps to a VPS host soon.