Skip to content

Dealing with “Apache (internal dummy connection)”

Today we had our first crash since we moved to the new server 7 months ago. Actually, the server wasn’t exactly crashed but it had come to a point where it was unresponsive enough to be unable to serve HTTP pages (and to make it a pain to connect to SSH). I wasn’t really in the mood so I didn’t take a screenshot, but top reported an impressive 90+ load average (NB: this server currently only has one core). Anyway, basically it seemed that the server was flooded with apache2 processes. Webmin reported a full use of our (massive ;)) 1 GiB of RAM as well as almost full use of our (twice as massive) virtual memory. I thought maybe we were being DOSed, although I don’t really see who would want to do that at the moment. I first stopped Apache, which basically solved the problem. But before starting it again I had a look at the logs… and in /var/log/apache2/access.log I found a massive amount of those: – – [27/Jul/2011:09:55:22 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:23 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:24 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:52 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:54 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:55 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:56 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:55:58 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:56:05 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)” – – [27/Jul/2011:09:56:06 +0200] “OPTIONS * HTTP/1.0” 200 136 “-” “Apache (internal dummy connection)”

I remember we already had issues with these internal dummy things on the previous server, but because software there was getting old our response to the problem was just, well, to move to the new server with all updated software and stuff. This time I decided to try to deal with the problem for real.

After some quick searches, I found out that internal dummy connections are used to wake up child processes. I’m not sure how all this works, but the key point here is that in the end, sometimes those calls may hit some of your “real” pages, and may also corrupt their cached version. Long story short, the request may end up more CPU-demanding than it should. That problem seems to have been solved in Apache 2.2.6, but although we are running a much later version I thought it wouldn’t hurt to still apply the work-around. Basically, all you need to do is to make sure that the dummy connections get a 403 error. So, just 2 lines in your configuration files (NB: it requires mod_rewrite to be enabled) (source):

RewriteCond %{HTTP_USER_AGENT} ^.*internal\ dummy\ connection.*$ [NC]
RewriteRule .* – [F,L]

And voilà, internal dummy connections should now be processed effortlessly. Don’t forget to apply configuration changes.

References (if you need more details you’ll find quite a few there):

Posted in servers.

6 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

  1. Justin Germino says

    Cool, I wonder if shared hosting providers allow these types of configuration changes for Apache?

  2. patheticcockroach says

    Usually in shared hosting you can’t change config files, but for those particular settings putting them in an .htaccess file should work fine. And anyway, this thing is only a problem for the resources used by the server, so probably your hosting provider will do their best to tweak it themselves!

  3. bbrian017 says

    I’ve done the fixes and I’m still crashing…. I’ve even managed to update apache to 2.2.22 but the issue still persists. My dedicated server i5 with 4 gigs of ram is crashing every night and I can’t seem to stop it. I’ve created this thread, to try and get some help, sadly I’m a marketer not a server admin but I’m trying to learn as I go. Seems updating apache using CentOS has it’s issues we had a hard time even getting it to 2.2.22

  4. patheticcockroach says

    Sorry I really don’t know better than what I wrote in this post. I’m currently using Apache 2.2.16 on Debian 6.0.4, and those dummy connections don’t appear in the logs anymore, without any special tweaking on my end (that’s a quite new install, so I remember doing nothing special about those dummy things)… That’s really some weird issue… :/

  5. hip0 says

    For me on Debian squeeze with apache2 .2.16 using the mod rewrite is not solving the annoying message 😐

  6. RiversideRocks says

    Man these are really annoying when they come up in the logs.

Some HTML is OK

or, reply to this post via trackback.

Sorry about the CAPTCHA that requires JS. If you really don't want to enable JS and still want to comment, you can send me your comment via e-mail and I'll post it for you.

Please solve the CAPTCHA below in order to fight spamWordPress CAPTCHA