Recent Topics

autoscript for .htaccess to block b2e blacklist domains

Started by on Jun 17, 2005 – Contents updated: Jun 17, 2005

Jun 17, 2005 16:53    

Has anyone written a script to automatically download the b2e blacklist file to stop referrer spam,
and to add the blacklist domains to an .htaccess file.
Seems to me that would be the easiest way to keep the .htaccess blocking up to date.

Barry

Jun 17, 2005 19:37

No, because the list doesn't just contain domains, it also contains keywords. I don't know if this could be done using .htaccess anyway.

Jun 17, 2005 21:42

Well, you can't do it with the Allow From/Deny From directives for that reason, but you could do it with RewriteRules.

I tried writing a script that would dump the whole list into a set of .htaccess rules, but my server has a limit on how many rules I can use, and it bogged down performance horribly as I approached the limit.

I've heard that a RewriteMap does the same thing with less of a performance hit, since Apache reads the whole map and then just does a lookup instead of crunching through 10,000 separate rules. However, I'm having a little trouble getting a RewriteMap implemented on my host. Anyone who knows a thing or two about it should check this out: http://www.totalchoicehosting.com/forums/index.php?showtopic=20511 and let me know if you have any suggestions. (Maybe there's just something wrong with my code?)

I'll of course make a post here and on my blog when I get this figgered out. They're wasting my bandwidth, too.

Jun 17, 2005 21:45

Also, you can make your life a bit easier by implementing [url=http://wonderwinds.com/hackblog.php/2005/02/07/antispam_recheck_tool_part3]EdB's antispam recheck tool[/url] and my [url=http://isaacschlueter.com/blog/automatic_antispam_update_cron]Antispam Update Cron[/url] technique.

I'm working on another cron script that will do recheck the top non-rechecked url. If that goes every few hours, then I really don't have to do anything to stay clean and up to date.

Jun 17, 2005 23:10

Ah, well, you can't use a RewriteMap in a .htaccess - that can only go in the httpd.conf file.

So that closes that possibility.

Jun 18, 2005 05:29

if I have removed all move referrers and stopped them from showing on my blog,
will the referrer spammer eventually leave me alone?

or what if I change the url of my blog, and don't have any referrers showing on the blog, will they not find my new address?

Barry

Jun 18, 2005 13:03

No and no. They'll try to hit you no matter what you do, since it's easier to jsut hit everyone than to differentiate between those who show referers and those who don't. And as to moving domains? Can you use google? So can they.

Jun 18, 2005 16:06

Is it just blogs that they hit?
I have other websites that don't get referrer spam at all.
Only my blogs.
My comments don't get hit, it is only referrer stuff.

I'm tempted to go with radio userland or something like that where I don't do the hosting

Barry

Jun 18, 2005 16:12

Well, that's up to you. I personally wouldn't sacrifice the flexibility I have with running my own server for having clean referer logs. I'm sure they look for the blog format, but I also have many sites that aren't blogs, are static html and get referer spam.

Jun 18, 2005 21:07

I've noticed that the referrers are all hitting the following page:

blog=2&disp=stats

I've disabled all display of stats
and all public links to stats,

can I turn off the function so the referrers get to a dead page with the above code....

my issue isn't clean referrer logs it is bandwidth usage

Jun 19, 2005 02:08

Crack this into your .htaccess file


# Try to prevent referrer spam
RewriteCond %{HTTP_REFERER} "!^http://your-domain.com/.*$" [NC]
RewriteCond %{QUERY_STRING} "disp=stats"
#RewriteRule .* - [F]
RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [R=301,L] 

Obviously you'll need to replace your-domain.com with your actual domain.


Form is loading...

Build your own site! – This forum is powered by b2evolution CMS, a complete engine for your website.