Update 12/8/2014: I’m not going to tell you not to cache all the things. You should absolutely cache all the things. However Otto wrote up a convincing argument about why you shouldn’t cache all the things using transients in the comments. So, when you’re done reading this post, go check the comments and see why what I’m suggesting is a bad idea. 😀
Let’s talk about transients.
Maybe you’ve been to a Wordcamp. Maybe you’ve been to a couple. Maybe you’ve never actually attended a Wordcamp but you read a lot of stuff about WordPress and how to make your site faster. Invariably, when discussing how to speed up your site, the issue of caching will come up and then things start to devolve into an unintelligible mass of arguments and conflicting opinions about what the best caching plugin is and strange words and phrases get thrown around like object cache and Pagespeed and SPDY and Batcache, Memcached and XCache…At this point, probably you just stop listening.
I’ll tell you a secret. My blog is slow as hell. I’m a WordPress developer and, yes, I will admit it, my site — running on a VPS — is slower than a Fail Whale eating a Tumblbeast for Thanksgiving. I went to all those “speed up your WordPress site” presentations — I even learned a couple things — and ultimately I’d come home, try a couple things, realize I needed to make complex server adjustments that were not intuitive, required me spending more time than I was willing to spend on, or required a level of control over the server and the configuration than I was allowed. I switched to Nginx for a while in an attempt to speed things up. I’d talk to tech support and do you know what they’d say? You have too many plugins installed. Try deactivating some plugins and see if the site speeds up.
At this point, I just stopped listening. I decided whatever, my site is slow, I’ll deal with it.
Caching
Caching is when you keep a copy of something so you aren’t requesting that thing when you load a webpage. In the olden days, this often meant building entire HTML files and storing them somewhere locally on the server and serving those files up as opposed to pulling content dynamically — in effect, making your dynamic WordPress site a static set of HTML documents. The problem with this is that producing these cached HTML files usually happened when someone loaded the page being cached, and if no cached file for that page existed, it would increase the load time for that page to create the cached file. Your browser keeps cached copies of webpages or resources (like CSS or JS files) from those pages you visit frequently, too, and this often means that when you go back to a site that has changed, you might need to flush your browser cache in order to see the live version that’s actually on the site.
Most caching plugins these days don’t serve up static HTML documents anymore (although some might) and, instead, they will use some form of server caching — possibly in conjunction with a Content Delivery Network — to cache large, memory-intensive database queries and serve large (or larger) javascript, CSS or image files from a fast server that’s not also responsible for delivering your website. This means that the caching plugin will often use the server’s own filesystem to store cached objects or data instead of dynamically pulling that data from the database when you load a page.
Now, somewhere along the way, you may have heard about object caching. Object caching is generally considered to be a good way of caching things just like caching things is generally considered to be a good thing to do. Object caching stores whatever it is that you are caching into your server’s memory, and only for the amount of time that you need it. It’s not good for persistent caching — when you are storing something for later — unless you are using a separate plugin, but on a high volume site, you might want to store a particular WordPress query into the object cache so all the data is right there when you actually try rendering the content on the page. The problem with object caching is your server needs to support it, and this requires that you know and understand your server configuration and are able to setup an environment where you could use the WordPress Object Cache class. In many cases and in some server environments, caching plugins can cause your site to run slower than it would without a caching plugin.
The ideal, though, at least for site speed, is having some kind of persistent cache — some way to save queries or pages that don’t change often (like less-actively-used tag archive pages, static WordPress pages or even single WordPress posts). Whether this is done by saving data to the server’s hard drive or using object caching — saving it to memory — doesn’t matter hugely, although it may come down to, again, what’s best for your environment. Many caching plugins will step in and use the WordPress Object Cache class for persistent caching, but you can’t assume that that’s the case, which limits the usefulness of these object caching functions.
Transients
Enter transients.
Plugin developers have been using transients for years and, let me tell you, transients are awesome. If you’ve ever felt the thrill as a Windows user hacking your registry to make one minor adjustment that saves you time or tweaks your system in a particularly rewarding way, you might relate to how cool transients are. Transients are like options you save to the WordPress options table but these settings can have an expiration. For example, if I’m a plugin developer, and I have a premium plugin, I might use a transient to store a user’s license key for a year. After the transient expires, the user might get a message that displays on the page that says it’s time for them to renew.
But an even cooler thing about transients is that they can be used to cache things like WordPress queries. A normal query you are running might look like this:
$paged = ( get_query_var( 'paged' ) ) ? get_query_var( 'paged' ) : 1;
$args = array(
'post_type' => 'post',
'paged' => $paged,
'orderby' => 'date',
'order' => 'DESC',
'post_status' => 'publish',
'ignore_sticky_posts' => 1
);
$my_query = new WP_Query( $args );
I can cache this query in a transient like this:
if ( false === ( $my_query = get_transient( 'my_cached_query' ) ) ) :
$paged = ( get_query_var( 'paged' ) ) ? get_query_var( 'paged' ) : 1;
$args = array(
'post_type' => 'paged',
'paged' => $paged,
'orderby' => 'date',
'order' => 'DESC',
'post_status' => 'publish',
'ignore_sticky_posts' => 1
);
$my_query = new WP_Query( $args );
set_transient( 'my_cached_query', $my_query, 12 * HOUR_IN_SECONDS );
endif;
It’s the exact same query. But before actually instantiating the WP_Query class, we’re checking against a transient. This transient is named “my_cached_query” and if it doesn’t exist — or if it’s expired — the $my_query = get_transient( 'my_cached_query' ) will return false and we’ll load the query normally. After instantiating the WP_Query with our arguments, we can save our query to the transient — remember it didn’t exist, that’s why we’re here — so it’ll be there next time, and I can give it an expiration — in this case I’m using 12 hours.
The next time this query is run, if it hasn’t expired yet, that conditional will return true — get_transient( 'my_cached_query' ) will exist, and so it will save to the $my_query variable. The next part — which would be your normal WordPress loop — would work normally. During that twelve hours, anytime someone runs that query, it’ll it should* pull from the cached WP_Query object in the transient rather than running that query directly.
Caveats
Now, obviously there are problems with this method. We don’t want to serve a cached query when there’s new content, so you’ll need to add a save_post hook to delete any transients that might be storing data that would be updated by the new content, otherwise you’d have a posts archive that doesn’t update except for once every 12 hours or so. And this is based on requests, it’s not run by a cron job, so 12 hours is somewhat relative to when the pages or queries or whatever it is that you are caching are actually executed — that’s when the transient would be checked and recreated if it had expired. And there’s also the fact that, as indicated with the * above, there’s a chance that the transient you set for 12 hours isn’t there when you expect it to be. Maybe there’s a plugin that’s flushing transients, in that case, you’d still be running the query every time.
On the other hand, if you are using a plugin that adds persistent object caching, “the transients functions will use the wp_cache functions” [Codex] added by the WordPress Object Cache class. In effect, the wp_cache_set and wp_cache_get functions are used in place the set_* and get_transient functions that you may have added in your code. And if there isn’t persistent caching set up on your site, transients will work normally and store in the WordPress database.
Transients are one of those things I never took time to really understand before and now, in retrospect, I wish I had. They are super easy to use, and once you do start using them, it’s easy to come up with a million and one possible uses for them. Even if you aren’t caching queries in transients, it’s still handy to learn the Transients API to temporarily store other bits of information that you might need later. It’s like a little sticky note on your computer reminding you to do something that you toss in the recycling after you’ve done whatever that note was reminding you to do.
re: “Transients are like options you save to the WordPress options table but these settings can have an expiration. ”
Actually, they are options, yes? It’s really just a question of where WP looks for them – in memory or in the DB.
Or have misread the codex? 🙂
They are options in that they are stored in the options table, but they are different because they have an expiration which usually options wouldn’t have (without building your own transient-like funcitonality with wp_cron or something).
I think it’s a good idea to point out here, that you don’t want to use transients in situations where the cache needs refreshed on a frequent basis. In that situation you may be better of using the raw wp_cache_get and wp_cache_set functions since they don’t default to the (slow) database when no persistent object caching backend is present. If you hit the database too hard, it may slow things down rather than speed them up.
Yep. Great point. It’s definitely worth learning the wp_cache_* functions. Transients are just a quick and dirty method. And I think wp_cache_get and wp_cache_set make more sense if you’re already familiar with getting and setting transients.
I don’t consider transients quick and dirty. Most of the time they are the best thing to use. They’re just not so appropriate for use with data which needs to be updated super frequently.
Please please please never use transients this way. The expiration for transients is a maximum life, and they are not guaranteed to last this long at all. They can (and do) get flushed more regularly, especially with object caches.
If what you want is an option with a set expiration, store the option and the expiration both as options. Anything else can disappear.
See this post for more on why, but basically, transients are for transient data. Don’t try and use them for anything else.
Point taken. Was just trying to give an example that people new to the idea of transients might be able to relate to.
One of the issues is that transients are misused a lot, which causes tonnes of broken behaviour. Not understanding that transients have a maximum lifetime is one of the worst mistakes, because it creates issues that can be near impossible to recreate locally: local environments typically don’t have the heavy object cache use that production environments do, so the least-recently-used cache isn’t as active.
One of the worst issues I’ve seen was a billing plugin that stored when subscriptions were due in transients. Upon activating object caching on the site as part of a performance review, all the existing subscriptions were immediately expired (as the system switched from reading from the DB to the object cache), and rebilled everyone in their system.
Misinformation is more harmful than no information, so please don’t encourage incorrect use. 🙂
“If all you have is a hammer, everything looks like a nail.” – Abraham Maslow, 1966
“So get a friggin’ screwdriver.” – Me, just now.
Don’t use transients for things that come from the database to begin with, like WP_Query results. This is just bad mojo.
Think of it like this: In a normal setup, you’re taking a bunch of information out of the database, and then storing it back in the database, and expecting a speed improvement. Madness, I say!
Instead, trust in the mySQL query optimizer and internal caching. Databases are designed to hold data in forms that can retrieve it quickly.. and they are smarter about it than you are. Really. Let mySQL do its job properly.
If you’re still having speed issues, then hacks like these aren’t the answer. Instead, invest your time in turning on a memory based cache. Memcache, XCache, APC, etc. Odds are that your host supports one of them somehow, or you can install one on your VPS, and there is very likely some existing plugin to connect the WordPress Object Cache to it directly. It just takes a minor amount of configuration and then magically everything becomes much faster.
For those not in the know, WordPress has “Object Caching” built in. It saves everything it gets into memory so that it doesn’t have to get it again. However, the default object cache is just in the PHP process. When every new request comes in, the object cache is empty. But if you have a persistemt memory cache somewhere else, and use a plugin to connect it, then now the object cache sticks around, and everything that previous WordPress runs got will still be there, available for it to use.
Even transients will use the object cache when there’s a memory cache hooked up to it properly.
See, here’s the query that runs from your basic WP_Query example:
SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts WHERE 1=1 AND wp_posts.post_type = ‘post’ AND ((wp_posts.post_status = ‘publish’)) ORDER BY wp_posts.post_date DESC LIMIT 0, 10
Now, what it is doing is to get the ID of all the posts that match your particular parameters. Later, it will go on to run other queries to get the content of those actual posts. That query will be something like this:
SELECT wp_posts.* FROM wp_posts WHERE ID IN (1,2,3)
And so on. Basically, get everything about the posts. Another query will happen after that to get things like taxonomy and meta information and such. All in all, it’s going to run about 5 database queries for a normal WP_Query. If it’s a complicated query, using tax_query or meta_query, than might be as high as 8-10 or so. All of them fairly fast, although some of those meta queries can be a bit tricky sometimes.
But the main query I’m concerned with here is that first one. It’s the one that will actually figure out which posts to get. It’s the one that will get those post IDs, and it’s the only one that runs every time.
Wait a minute, don’t they all run every time? Not with persistent object caching they don’t.
For every one of those other queries which is actually retrieving data, they first check the object cache. So, if I’m getting post numbers 1, 2, and 3, and 1 and 3 are already in the object cache, then I only really have to get post 2. Time saved. If they’re all there, then I don’t have to make any query at all. Taxonomy info, meta info, these are all object cached too. Same principle applies.
Now, you might be saying that if I have a persistent memory solution, then the transients will use it too, and my transients go there. This is true, but inefficent. The way memory caches work is that they are of limited size, and when it fills up, stuff that hasn’t been accessed in a while gets dropped. Here, you’re caching the whole query in a transient. All the results in one big blob, sort of thing. If you just performed the normal query, then each post and taxonomy and everything else is a separate thing in the object cache. The query simply gets them all individually and combines them.
Smaller items in the cache = less chance of them expiring early due to memory limitations.
The short of it is that transients don’t make any sense for wrapping around database queries.. You use them for wrapping around things that take too long in other ways. When you have to process a bunch of data all at once and build a big table which doesn’t change often, or when you have to get data from another server using http and that other server only needs to be queried once a day. Things like that. Then it makes sense. But the database being slow to return data to you just isn’t a really good use-case. Instead, you need to optimize your queries so that the database is fast, or use a faster database server, or use a memory based object cache so that you don’t have to query the database nearly as often anyway.
You are somewhat correct. But some queries are just enormous, and caching them makes things run a ton faster then.
MySQL does internal caching, but it doesn’t necessarily know that you are happy to keep the same data kicking around for 10 minutes, so it’ll go grab it fresh each time, which can suck performance wise if it’s a huge query. Transients allow you to work around that.
Ryan, if you store the data in a transient, and transients are in the options table (the default), then you’re actually grabbing *more* data each time, because his example shows storing the whole WP_Query object in there. Much larger than simply getting the rows you want.
If mySQL is being slow, then addressing that by shoving more data into it in a big ugly non-relational field doesn’t seem like a great solution to me. Yes, you brought your 5 fast indexed queries down to 2 fast indexed queries, but really, if it’s that slow to begin with, then addressing the root of the problem seems like a better long term strategy.
Storing database query results in a transient is a hack. Plain and simple. Hacks often work, no argument. But profiling the performance and then addressing the actual bottleneck lasts a whole lot longer.
If the query is sufficiently complicated, then optimising it isn’t going to help. Sometimes it is more efficient to store a big ugly data blob than it is to perform a complicated query each time.
If your query really gets that complex, then maybe it’s time to move to a better data model. The WordPress model fits a lot of cases. Not all cases.
I see two options there, change to a better data model, or cache it in a transient. Either way results in fast page loads, one way takes less effort. I think I’ll stick with using less effort 😉 (that and I suck at query optimisation)
You guys are awesome 🙂
Otto, your comment makes perfect sense, thanks for taking the time to write it out. 🙂
Oh, also, because you’re using a transient with a fixed name, your “paged” parameter won’t work anymore. The first page will have your results, and the second page will have the same results because you got the results from a transient instead of running a new query to get the next page of results.
See, it’s just bad mojo, I’m telling you. 🙂
You can use transients in paged results without any issues.
Carlos
$paged = ( get_query_var( 'paged' ) ) ? get_query_var( 'paged' ) : 1;
$my_query = get_transient( 'my_cached_query_$paged' ) ;
If ( false === $my_query ) {
/**
* My query goes here
**/
set_transient( ''my_cached_query_$paged' , $my_query, 12 * HOUR_IN_SECONDS );
}
(Please excuse messy syntax)
> We don’t want to serve a cached query when there’s new content, so you’ll need to add a save_posts hook to delete any transients
I think you mean `save_post` hook, not `save_posts`?
https://codex.wordpress.org/Plugin_API/Action_Reference/save_post
Yep. Typo. Thanks for catching that.
I have a use case where I need transients or something like transients to reduce the amount of load on the server, I would love to know if you think transients are going to be useful in this situation.
Im working on a job where I am using php functions to collect external data that is stored into cpt fields(Server cron executes the function every 3 minutes and updates data). The front end user then uses wp queries to collect and sort the cpt entries, those results are then looped and html is produced. On top of that I am using ajax on a time interval to call the function that contains the wp queries and the result is redelivered to the same session without needing a refresh. All of this has been written into a custom plugin.
Do you think I will benefit much from using transients to save the query results or is there something more suitable that I should be looking at? Appreciate any feedback, this post has been really informative 🙂
Using transients to store data/queries from an external API/source is the perfect time to use transients. You don’t want to hit that API with every request both because of the time lag to connect to the API and because many APIs have a maximum number of requests, and a lot of times the data isn’t going to change as frequently as you might be making those requests anyway.
i’m using transient for cache present content and ready for new contents.but i have a problem with delete transient cache.when new posts publish,new posts won’t display!!
Hi Javad- It sounds like you may not be deleting the transient when new posts are saved. You can do that by using the WordPress ‘save_post’ action hook, like in this example: https://gist.github.com/kellenmace/33f26860a720dd4cf1b8b84716c9fe97. I hope that helps out. Thanks for reading 🙂
Now what I need to know is how do I use transients for the kind of caching explained here. Is there a separate plugin or something which can be used to achieve this functionality?