Apache's 'internal dummy connection' [21082007]
Since version 2.2, the ubiquitous Apache webserver has begun intermittently issuing GET requests to itself. They hit the root '/' url. They appear in your access logs looking something like this:
::1 - - [...] "GET / HTTP/1.0" 403 16 "-" "Apache/2.2.4 (Unix) mod_ssl/2.2.4OpenSSL/0.9.8a DAV/2 (internal dummy connection)"
And in your Rails logs like this:
Processing Controller#index (for ::1 at ...) [GET] <snip> 200 OK [http://127.0.0.1/]
Fine, GETs should be idempotent, so it's completely safe practice and don't you worry, son. Well, except...
Here's a common web application pattern wherein GETs are not quite safe: caching on demand. When a resource is requested, the web server is instructed to check whether a cached version of the resource exists (eg, a static .html page). If it does, it is served up. Otherwise, a web application is invoked and asked to serve the page. The web application does so, and saves a cached version for future requests. (It will have some internal or external logic that determines the expiry of cached resources.)
So there's a scenario where a request might be issued and the result saved for all future requests. What if that request comes from a local, rather than a remote, caller? This can have a bearing on the response. In the case of Ruby on Rails, urls constructed with url_for() use the local domain (127.0.0.1) rather than the remote domain (example.com).
If the response to an internal request is cached, it's going to get served up to all your remote clients. And all those localhost links aren't going to work for them. This is the problem we have with Apache's 'internal dummy connections': they intermittently corrupt our cache.
I deal with it rather bluntly:
RewriteCond %{HTTP_USER_AGENT} ^.*internal\ dummy\ connection.*$ [NC]
RewriteRule .* - [F,L]
This mod_rewrite snippet, which belongs in your .htaccess file or <VirtualHost> config, sends the internal connections a 403, keeping them well away from your application.
Well, the scant info on the web suggests that these requests are used by Apache to tell its child processes to die (when a graceful restart is issued, for example). So it's not a reshaping of an external request, but a genuine internal request.
Still, its effects on Rails' caching patterns are very real, not least because cache flushes often coincide with graceful restarts in one's deployment scripts.
Thanks, your blog helped me to ignore this error.
Thanks Buddy
Thanks Joseph!.. This has been corrupting my cache files in our cakePHP based CMS .. very useful post all these years on!
Joseph Pearson is a founding Software Inventor at Inventive Labs.
How is apache justified in falsifying where the request is coming from (and going to)? Why would it expect to work ever?
To me, this has nothing to do with caching or idempotent. A get requested from A of page Foo, is not the same as a page requested from B of page Foo. That assumption is just false.
A may be trusted. B may not be.
This looks like the beginning of a security hole to me.