Update 16th Sep: Scroll down to comments for some technical info about moving to InnoDB

Drupal has definitely matured into an enterprise ready CMS. With this growth has come a need for high performance solutions for sites that weal with high visitor traffic. This is where Pressflow really shines - Pressflow is a Drupal distribution created by Four Kitchens that aims to deliver high performance to Drupal users and webmasters by streamlining some of the bottlenecks found in core Drupal. We strongly recommend it to developers and system administrators who are facing heavy traffic. In fact, we now base all of our projects on the Pressflow version of Drupal.

Some of the highlights of Pressflow’s improvements are that it comes with support for slave MySQL servers, implements a better locking framework, recommends InnoDB, supports lazy session handling, to name a few. In fact many of these features are so useful that they made it to Drupal 7 core.

One of our client’s sites suffered from a major growth in traffic over the last few months, going from about 3 million to 10 million page views a month. In order to deal with this higher traffic we decided to move from Drupal vanilla core and MyISAM to Pressflow and InnoDB. The migration was complete last weekend and the results are simply incredible. We expect even further improvement since we can now also implement Varnish caching in the near future.

Statistics

Since a picture is worth a thousand words we‘ll share some of the server monitoring graphs in order to give you a more detailed understanding of the migration and its performance results.

The main bottleneck in our case was that the database server was bogged down due to certain Views queries, the most complex of which had several JOINs. The database server wasn’t able to handle the traffic peaks and would often become overloaded with loads exceeding 50.

Slow queries

The slow queries in all cases simply disappeared. Day 12 is when we migrated to InnoDB, and after that the number of slow queries is zero.

Memory usage

Following chart shows the database server’s memory usage. Migration to InnoDB allowed us to better utilize the 16GB of the memory.

CPU & Load

What about CPU usage? You can see from the CPU usage graph there are no more spikes, which were causing the server to be unresponsive.

The same dramatic improvement is shown by the average load of the server.

Importance of monitoring

The last chart I’d like to share with you shows something important. The previous charts all show dramatic improvement and most sysadmins would be happy to leave the server now with its improved performance.

We always look deeper to find what will be the next issues to make sure that we are ready for bottlenecks before they appear. When we were finishing our analysis, we saw that around day 10, which was when we moved to Pressflow, that there was a sudden spike in INSERT queries. If sysadmins are not used to reading logs and monitoring, they probably won’t know what is actually behind the monitoring statistics and wouldn’t notice this spike. After poking around a bit we found that Pressflow changed how error_reporting behaves. These INSERT queries were inserts into watchdog because the client’s site had some notices and undefined indexes. This database load issue would have been mitigated by using Syslog module instead of DBLog, which is definitely more appropriate on a site with this much traffic. Still though, there would be an onslaught of error messages to wade through. We deployed a quick hot fix by changing the PHP configuration error reporting levels and made a ticket in our project management system to deal with the the notices later in a more permanent way.

The lesson

What is the lesson that we learned?

1. Seriously consider Pressflow if you require truly high performance from Drupal.
2. Don’t be afraid to move to InnoDB, in most cases, it will deliver better performance (Drupal 7 defaults to it). A quick test on your development server won’t take too much time.
3. Server administrators and Drupal developers need more then just deep knowledge of the technologies they use on a regular basis - they need to be passionate about their work and understand the big picture about Drupal and high performance web site hosting.

Thanks to Andrew Burcin for helping with this post.