Recent Topics

1 Oct 01, 2005 15:29    

Hi Gang,
over the past week or so, I've been keeping an eye on my site's error logs, and they're full of stuff exactly like this.

[Sat Oct  1 08:00:25 2005] [error] [client 208.50.100.1] File does not exist: /home/lenwood/public_html/404.shtml
[Sat Oct  1 08:00:25 2005] [error] [client 208.50.100.1] File does not exist: /home/lenwood/public_html/htsrv/trackback.php
[Sat Oct  1 07:50:41 2005] [error] [client 66.196.101.96] File does not exist: /home/lenwood/public_html/404.shtml
[Sat Oct  1 07:50:41 2005] [error] [client 66.196.101.96] File does not exist: /home/lenwood/public_html/robots.txt
[Sat Oct  1 07:41:05 2005] [error] [client 68.142.251.174] File does not exist: /home/lenwood/public_html/404.shtml
[Sat Oct  1 07:41:05 2005] [error] [client 68.142.251.174] File does not exist: /home/lenwood/public_html/robots.txt
[Sat Oct  1 07:27:38 2005] [error] [client 69.95.84.110] File does not exist: /home/lenwood/public_html/404.shtml

I'm wondering about the security threat that these searches pose. Why would they be trying to directly access a 404 page? Or 'robots.txt'? Should I be concerned about this, or is it not a big deal?

I think the thing that concerns me most about this is the volume. My error log holds the 300 most recent errors, and each time I check it the oldest entry is no more than 6 or 7 hours old. They're at it all the time!

Any input on this will be appreciated.

Thanks,
chris

2 Oct 01, 2005 21:10

ok.

[Sat Oct 1 07:41:05 2005] [error] [client 68.142.251.174] File does not exist: /home/lenwood/public_html/robots.txt

68.142.251.174 == lj2564.inktomisearch.com == bot
A bot looking for a robots.txt is normal.

[Sat Oct 1 08:00:25 2005] [error] [client 208.50.100.1] File does not exist: /home/lenwood/public_html/404.shtml
[Sat Oct 1 08:00:25 2005] [error] [client 208.50.100.1] File does not exist: /home/lenwood/public_html/htsrv/trackback.php

208.50.100.1 has an open proxy, and since they tried to access your trackback.php, that was prolly a spam attempt.

etc...

as for the 404 attempts, those arent direct calls. theyre the results of other pages not being found -- hence apache looks for a 404 page, doesnt find one, and tells you so.

robots.txt is not going to be something the average human is going to find anything useful in.

http://network-tools.com/default.asp is an easy place to reverse any ips you might be curious about.

Would I worry about what you pasted? No.

If you are interested in lessening the errors, create a robots.txt file and 404 page : http://www.thesitewizard.com/archive/custom404.shtml.

Youre looking at your error log in cpanel (last 300 errors) and with things like that not taken care of, its fairly normal to have that busy. Another common thing ppl see is favicon.ico

3 Oct 01, 2005 23:15

Thanks whoo. I've created both a robots.txt file, and a 404 page. They're both super simple, it took less than 10 minutes to create both. Any recommendations on what I should disallow in my robots.txt? I blocked access to cgi-bin, admin and b2evocore.

Thanks,
Chris

4 Oct 02, 2005 01:02

nope not really :) someone else might though.

I personally try to minimize any site caused entries in that log just so I know that what I do see in there is the result of something "they" have done. I would just deal with things as they come up B)


Form is loading...