WordPress + Amazon S3

PLEASE NOTE: This post was originally created in 2011 and some of the processes and plugins mentioned may no longer be correct or available.

Recently I’ve been tasked with management of a couple of WordPress websites that get more traffic than I’m used to dealing with. Quite a bit more in fact. So much so that they were causing the server they lived on to have memory issues, random reboots and other odd issues.

Step 1

In order to get the sites more stable, I started clearing out plug-ins. Starting by deleting the ones that were being used and actually deleting them off the server. Then removing plugins that weren’t really needed and replacing them with a WordPress function where possible.

Result: Not enough to stablize the server

Step 2

Looking at the logs, I found quite a bit of traffic to the WordPress comment system. These sites are running Disqus, so WordPress comments aren’t needed. Got rid of them. I also noticed quite a bit of traffic and strain on the servers coming from other sites, hotlinking to images, javascript files and content. Turned on hot-link protection for all but a few sites and blocked a few IPs.

Result: Better, but server still not 100% stable.

Step 3

Looking at the remaining plugins and with help from support, the next culprit was the NextGen Gallery plugin. This is a key component on each site and disabling it actually was a bad thing. But I had a server that was rebooting itself every day, so disabling it I had to do.

Support was also able to temporarily upgrade the server (it’s a virtual server, so easy for them to do).

Result: Stable server!

Step 4

Needing to allowing more hotlinking than I had setup, NextGen gallery support and to improve the overall speed of the sites, it was time to look for solutions. It was recommended to look at using a Content Delivery Network (CDN) like Rackspace or Amazon S3. Having not used a CDN before, I figured it was as good a time as any to give it a whirl, and I decided (based primarily on price) to use Amazon S3.

Starting here at donburnside.com.

The google pointed me at a few plugins that have been crucial in making the switch to S3 possible.

  • Creating a CNAME DNS entry for the S3 bucket.
  • WP Super Cache. Some recommended W3 Total Cache, but I read it gave memory errors on occasion (a problem I was trying to escape) and Super Cache was already installed on both sites
  • Amazon S3 for WordPress. What this does is take anything that is uploaded using the WordPress media uploader and automatically moves it into the S3 bucket of my choosing.
  • CDN Sync Tool. While it acts weird on my server because a depreciated piece of PHP is missing, I was still able to make it work to get all of the data needed over to the Amazon S3 bucket and keep them updated, kind of.

Starting with the CNAME entry, I mapped it to an identically named S3 bucket. This is needed to make sure everything works correctly. The CNAME entry is required for the Amazon S3 plugin to work correctly.

Once that was setup, I used CDN Sync to get all of the necessary files to the bucket. It’s also used to keep certain directories updated, like the gallery directory. That part happens manually which I will discuss later.

Finally, turning on CDN support in Total Cache. This redirects any call to any folder specified (wp-content and wp-includes are default) to the CNAME’d entry that I previously specified.

Done and done. WordPress, within a few hours, running from a CDN. Since I’ve not done this before, it took much longer than a few hours. Now that I know how easy it is, it won’t next time.

NextGen Gallery + Amazon S3

Currently, there is no support for any CDN within the NextGen Gallery Plugin. This is a problem because, like I mentioned, the sites rely heavily on that plugin. Some more googling brings me to this thread at the WordPress.org support forums offering a hack to redirect the plugin to look at the S3 bucket I needed. And it works very well with 1 minor hitch. Hacking a plugin to work outside of it’s normal parameters means with the next update, your hack will be gone. So I created a duplicate of the file that I modified so I can replace it (or duplicate the hack) after an update.

Once that is done, when you upload images to the gallery they go into the gallery folder on the server just like always, but you can’t see any of the images or thumbnails since they are being referenced back to the CDN. That means that once a gallery is created and images are uploaded, you have to go back into CDN Sync Tool and run a sync on the wp-content/gallery/[directory name here]. That will move all of the images and thumbnails to the CDN and you can post the gallery, serving images from the CDN.

Now I have sites that are not only running faster than before, but they are using half as much bandwidth from the hosting company and fewer resources (all js and css files are also being served from the CDN). And, at least on Amazon, for not very much cost. On 2 sites that get approximately 175K views per month, the cost is going to be about $30 or less, and the bandwidth savings on the server is close to 75%!

If you are looking for a way to speed up a site or reduce the load on a server, I can not recommend this enough.

Comments are open if you have any questions!

By Don

Lead bottle washer at donburnside.com, host at whiteroofradio.com and tech guru for the MotoringFile family of sites.

10 comments

    1. I’m sorry, I don’t understand what you mean. If you are looking for an update to the NextGen Gallery, you would want to talk to them about it.

      Also, I’m not longer attempting to run NextGen in this fashion. Instead, I created my my own gallery system that works very well. I’ll be posting about it very soon!

  1. CDN Sync tool is not great. It doesn’t allow you to set a persistent set of extra folders in a list to sync. So if you have multiple galleries in nextgen gallery you have to continually update them as you edit them. It also doesn’t appear to have an option to copy all folders under a given folder which would be really useful.

  2. Hi Don, Interesting post. I also need something so my nextgen galleries are not on the hosting server but on S3 or even maybe dropbox.

    I see you created your own gallery system, did you ever release it?

    1. Sorry I didn’t. It only worked on themes of my own creation and Twenty Ten. and I wasn’t able to make the navigation work 100%. It was too fiddly.

      I have started creating custom gallery post types to keep photo galleries organized and using Amazon. Add a little extra styling in your functions.php to make it look a bit different if you like. Or don’t. And, add a lightbox plug-in. It’s almost as good.

      I might revisit the gallery system at a later date.

      Thanks Darren!

  3. Wonderful post! Thanks for the useful tips. Kindly check this one http://wordpress.org/plugins/iframe-embed-for-momentme/. An
    easy to use plugin that works fast. It creates a 360° perspective of your
    images. Great for enhancing your web content. It also collects publicly
    available photos and images of social event automatically. And the best thing
    is, it’s free.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.