Kernel - slooooow??

[lellie]

Is there a problem on kernel again? Just it's very very slow to serve up any pages at the moment.. mail seems fine but it keeps timing out when I'm trying to load any of my sites (all of which are on kernal..)

[Matt]

When did you start to experience these problems and were they PHP pages, straight HTML, or what? I received a similar report yesterday where the normal http://kernel.apisnetworks.com/ URL loaded up quickly and the others were slow on the server.

It may've been linked back to the fact that keepalives and timeouts were set extremely low and have not been raised (up until today).

[Matt]

Actually I managed to experience earlier this afternoon what you were hinting on the other day about performance. Turns out to have been one last zombie machine SYN flooding Kernel from the initial attacks. I've added that IP address to the list of hosts to drop from the server and everything looks ok now.

[lellie]

I'm still suffering here.

the pages are PHP.. they keep timing out and it's been completely down on sonance.co.uk for much of the week..

In general it's been a bit rubbish really - especially as the most important 2 weeks of the year are on the end of this month for promoting the site (university freshers weeks in the UK)..

[Matt]

I'm still suffering here.

the pages are PHP.. they keep timing out and it's been completely down on sonance.co.uk for much of the week..

In general it's been a bit rubbish really - especially as the most important 2 weeks of the year are on the end of this month for promoting the site (university freshers weeks in the UK)..

Do a traceroute for me to the server's IP address. I've not heard nor seen any further problems since last week. Also give me your IP address so I can do a reverse traceroute to you from the server. Latency may be to blame here.

Edit: found the problem. The script responsible for blocking IPs picked up Kernel's IP address during one sweep as a potential flood and blocked connections going to itself over TCP/IP. You probably had an include('http://...') or an fsockopen() call in there causing the server to poll itself creating the problem.

[lellie]

Do a traceroute for me to the server's IP address. I've not heard nor seen any further problems since last week. Also give me your IP address so I can do a reverse traceroute to you from the server. Latency may be to blame here.

*noob* You'll have to explain what I need to do..
my ip is 84.12.110.133

[Matt]

*noob* You'll have to explain what I need to do..
my ip is 84.12.110.133

There's no need to now. I isolated and resolved the problem (see above).

[lellie]

also other users have been experiencing problems too (on different ISPs etc)

[lellie]

also other users have been experiencing problems too (on different ISPs etc)

ah yeah I do have some includes.. :) ta.

[lellie]

Some of my php includes are still timing out.. but not causing the whole site to crash now so not quite so bad.

[Matt]

Some of my php includes are still timing out.. but not causing the whole site to crash now so not quite so bad.

Oops, sorry about that. Watchdog kicked in last night after one fellow's jBoss application puked and generated about 6000 notice e-mails in the span of about 10 minutes, spiking load and forcing a reboot. I've removed the iptables block of kernel from the save so this shouldn't happen again in the future.

[lellie]

hehe you're not having a good time of it at the moment! Seems to be one thing after another..

I'm still getting downtime.. everything was fine until a couple weeks after we moved to kernel from sprite and since then it's been so unreliable.. :(

Thanks though - you do try and sort things out nice and quickly :)

[Matt]

hehe you're not having a good time of it at the moment! Seems to be one thing after another..

I'm still getting downtime.. everything was fine until a couple weeks after we moved to kernel from sprite and since then it's been so unreliable.. :(

Thanks though - you do try and sort things out nice and quickly :)

Duplicate entry for the server's IP address in iptables. I only removed one in the chain and the other still remained. Definitely fixed now and I double checked your Web site. My apologies for the delay in resolution of it.