W3TC is a page cache which uses obstart() and grabs the entire page, writes it to disk or memcached and then outputs it. The header rotator outputs html which is inserted in the page. When you’re logged in, Varnish wouldn’t actually cache the index page, allowing the img src which rotates the header to be generated on each pageload.

W3TC’s config options are very confusing and without testing and retesting, it is very difficult to find the best configuration. WP Super Cache’s out of the box setup is really much better than W3TC’s. With enough time, you could find the magic combination of settings to be faster than WP Super Cache. There are other benefits to W3TC that can be duplicated with other plugins, but, if you wanted everything in one place, W3TC contains a number of useful plugins all in one place.

The other thing you need to consider with your benchmark is that Linode uses virtio on the interface and benchmarking on the machine without actually using the network is going to allow you to have considerably higher numbers than if you’re actually moving traffic across the network. You can tell W3TC to not cache for logged in users which more closely replicates how Varnish is handling things. Also, I would suspect that on non-logged in users, the images wouldn’t be rotating until the cache time expired or the page was purged from the cache on both W3TC and Varnish – and, as your latest comment suggests, WP Super Cache.

Doubled the performance of W3TC and Varnish with 10,602 requests/s.
The load for non cached pages feels noticeably quicker.

This is all on the smallest Linode VPS.

I’ve just tested the following on a default WordPress 3 install:
- bare bones Nginx
- Nginx + W3TC (page cache and minify using disk enhanced)
- Nginx + Varnish (removed all cookies) + WP Varnish plugin

All of these have APC opcode cache enabled.

The bottom 2 configs are producing very similar results, but W3TC is more stable. My ab test for Varnish hangs on the first try and also slows down after I repeat the test many times. I’ve checked memory and there’s plenty free.

Also, in terms of user experience, there’s a big banner image on the default WordPress theme that rotates randomly. W3TC caches that page and the same banner image shows. Varnish is letting the home page image reload even though I’ve removed all cookies in the VCL. I wonder if it’s to do with browser cache settings (will investigate).

I saw a comment from Frederick Townes somewhere that he’s lost interest in Varnish. I wonder if that’s based on his own tests.

As for my results:
- bare bones Nginx = 57 req/s
- Nginx + W3TC = 5180 req/s
- Nginx + Varnish = 5124 req/s

I wanted to have a nice clean Nginx + Varnish solution (incl Plugin), maybe with a minify plugin, but that’s it. But the Varnish results aren’t especially impressive for the additionally complexity it adds.

Any thoughts on anything I’m missing regarding Varnish? My configs are really bare bones. No funny business.