Spam: an IP based approach
Goto page Previous  1, 2
 
Post new topic   Reply to topic   printer-friendly view    b2evolution Forum Index -> Plugins & Extensions -> Fighting spam!
View previous topic :: View next topic  
Author Message
mweis
New Poster
New Poster

Joined: 27 Jul 2005
Posts: 5
Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Fri Aug 12, 2005 4:14    Post subject: Reply with quote

keninman wrote:

I am thinking of password protecting all of my blogs and requiring anyone who wants access to at least use a generic username and password to enter. Kind of like driverguide does to keep bots out. If anyone has done this successfully please let me know.


This sounds like a good idea to me, I have another observation which might track spammers: they access the stats page regularly (URL...&disp=stats) to see, if they are successful. This might help to track spammers. If they access the page -bang/trapped- never again, please.

Since I have no implementation for this so far, it is only an idea.
In index.php this is my first thing to do now:
Code:

// the very first thing: check for spammers
 if (strpos($REQUEST_URI,'disp=stats')) {
                                      // this should redirect them home, if uncommented
                                      //header('Location: ' . $HTTP_REFERER);
                                      die ('die, spammers!');
 }
/**
 * First thing: Do the minimal initializations required for b2evo:
 */
Back to top
View user's profile Send private message
keninman
New Poster
New Poster

Joined: 10 Aug 2005
Posts: 8
Reputation: 9.1Reputation: 9.1Reputation: 9.1Reputation: 9.1Reputation: 9.1Reputation: 9.1Reputation: 9.1Reputation: 9.1Reputation: 9.1 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Fri Aug 12, 2005 12:29    Post subject: Reply with quote

Quote:
// the very first thing: check for spammers
if (strpos($REQUEST_URI,'disp=stats')) {
// this should redirect them home, if uncommented
header('Location: ' . $HTTP_REFERER);
die ('die, spammers!');
}
/**
*/


This is what I pasted does it look right?
Back to top
View user's profile Send private message
EdB
/bb|[^b]{2}/

Joined: 05 Jan 2004
Posts: 7123
Reputation: 140.9 add or subtract from this member's reputationadd or subtract from this member's reputation
votes: 90

PostPosted: Fri Aug 12, 2005 12:51    Post subject: Reply with quote

Please start a new thread for this new topic (password protecting the entire blog). Someone else might be interested in it and NOT find it because they are not interested in an IP-based solution.
Back to top
View user's profile Send private message
gloin
New Poster
New Poster

Joined: 07 Mar 2005
Posts: 27
Reputation: 60.1Reputation: 60.1Reputation: 60.1Reputation: 60.1Reputation: 60.1Reputation: 60.1 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Tue Aug 16, 2005 1:57    Post subject: Holy Crap! I love it! Reply with quote

This is the real deal here. I implemented the above, and am laughing my arse off watching the output of "tail -f /var/log/apache/access.log" as it's nothing but 302 after 302 after 302! Until, of course, I connect to my own blog, at which time I see a happy 200.

I had to do everything I could as the spam, even though blacklisted, was thrashing the hell out of the mysql server on my linode. I'd catch the server apparently dead during a spamrun, and find out that I was 190 megs into swap. And the spammers are using large botnets now, so I couldn't manage all the iptables rulesets effectively in my spare time. This, however, is entertaining and effective.

Thanks a kazillion!
Back to top
View user's profile Send private message
mweis
New Poster
New Poster

Joined: 27 Jul 2005
Posts: 5
Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Tue Aug 16, 2005 12:39    Post subject: Re: Holy Crap! I love it! Reply with quote

gloin wrote:
...watching the output of "tail -f /var/log/apache/access.log" as it's nothing but 302 after 302 after 302! Until, of course, I connect to my own blog, at which time I see a happy 200...


As long as you don't want to see your stats. It is only half of the traffic you can stop this way (only the checks).

Since this is about IP-based approaches I took the idea into blocking them via .htaccess. If you are interested in the IP-numbers i block, have a look at http://datenroulette.de/blog/?disp=spammers

What is it? I track the IPs in the database, and if they come back 4 times -bang- update .htaccess. This should stop any traffic from the bots.

Here is this (quite ugly) hack, I put it directly into index.php, but maybe it is good to have it in hacks.php as a function. However, here it is:
Code:

 
/*create table (phpmyadmin or mysql), key is necessary for the 4.1.x syntax
CREATE TABLE `spam_ips` (
`ip` TINYTEXT NOT NULL ,
`count` INT NOT NULL ,
PRIMARY KEY ( ip( 15 ) )
) TYPE = MYISAM
ALTER TABLE `spam_ips` ADD `banned` TINYINT( 1 ) ;
*/
$spam_user="db_user";
$spam_pass="db_pass";
$spam_host="localhost";
$spam_db="db_name";
$spam_table="spam_ips";
$htaccess_file='/home/your_userdir/public_html/blog/.htaccess';

 if (strpos($REQUEST_URI,'disp=stats')) {
 //log their IPs
      
   $link = mysql_connect($spam_host,$spam_user,$spam_pass)
   or die("Keine Verbindung möglich: " . mysql_error('cn'));
   mysql_select_db($spam_db) or die('db');
   
   /* ausführen einer SQL Anfrage */
   /* With mysql >= 4.1.0 you can insert ... ON DUPLICATE KEY update
 If you specify the ON DUPLICATE KEY update clause (new in mysql 4.1.0), and a row is inserted that would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an update of the old row is performed.
 */
 //these are the basic queries, if you use mysql v.4.1.x you should simply use teh first query instead of the if-else construct
 $mysql4_1_x_query="INSERT INTO `".$spam_table."` (ip,count) VALUES ('".$_SERVER['REMOTE_ADDR']."',1)
 ON DUPLICATE KEY UPDATE count=count+1";
 $selquery="SELECT `ip`, `count` FROM `".$spam_table."`  WHERE `ip` = '".$_SERVER['REMOTE_ADDR']."'";
   $insquery = "INSERT INTO `".$spam_table."` SET `ip` = '".$_SERVER['REMOTE_ADDR']."', `count` = 1";
   $updquery =   "UPDATE  `".$spam_table."` SET count=count+1 WHERE ip='".$_SERVER['REMOTE_ADDR']."';";
   $result = mysql_query($selquery) or die('qr');
   
 // here is the code to update .htaccess
 $line = mysql_fetch_array($result, MYSQL_ASSOC);
 //var_dump($line);
 if ($line['banned']==NULL && $line['count']>2){
  if (is_writable($htaccess_file)){//echo 'banned';
  $filehnd=fopen ($htaccess_file,'a');
  fwrite($filehnd,'Deny from '.$line['ip']."\n");
  fclose($filehnd);
  $updquery =   "UPDATE  `".$spam_table."` SET banned=1 WHERE ip='".$_SERVER['REMOTE_ADDR']."';";
  mysql_query($updquery) or die('qru');
  }
  else{echo 'file not writeable ';}
 }
 if (mysql_num_rows($result)>0){
  $result = mysql_query($updquery) or die('qru');
   }
   else{
      $result = mysql_query($insquery) or die('qri');
   }
   //mysql_free_result($result);
      die ('die, spammers!');
 
 }

 // show spamips, if requested via disp=spammers
 if (strpos($REQUEST_URI,'disp=spammers')) {
   $link = mysql_connect($spam_host,$spam_user,$spam_pass)
   or die("no conn");
   mysql_select_db($spam_db) or die('oops'.mysql_error());
   $selquery="SELECT `ip`, `count`, `banned` FROM `spam_ips` ORDER BY `count` DESC";
   $result = mysql_query($selquery) or die('outch');
   /* Ausgabe der Ergebnisse in HTML */
   echo "<table border=\"1\" width=\"100%\">\n".
      '<tr><th>ip</th><th>count</th><th>banned</th></tr>';
   while ($line = mysql_fetch_array($result, MYSQL_ASSOC)) {
   //var_dump($line);
   echo '<tr><td>'.$line['ip'].'</td><td>'.$line['count'].'</td><td>'.$line['banned'].'</td></tr>';
   $denystring.='Deny from '.$line['ip'].'<br>';
   }   
echo '</table>';
echo 'These are the spammers IPs <br>';
echo 'For your .htaccess:<br>'.$denystring;
die ();
}

/**
 * First thing: Do the minimal initializations required for b2evo:
 */


make shure your .htaccess is writable to the webserver.

A problem might be that the file will grow very large. We will see.[/code][/url]

This is the result in the error log:
Quote:

[Wed Aug 17 11:09:44 2005] [error] [client 216.86.156.205] client denied by server configuration: /home/datenrou/public_html/blog/index.php
[Wed Aug 17 11:08:37 2005] [error] [client 216.86.156.205] client denied by server configuration: /home/datenrou/public_html/blog/index.php
[Wed Aug 17 11:08:12 2005] [error] [client 207.248.240.118] client denied by server configuration: /home/datenrou/public_html/blog/index.php
[Wed Aug 17 11:08:07 2005] [error] [client 62.77.41.21] client denied by server configuration: /home/datenrou/public_html/blog/index.php
[Wed Aug 17 11:08:07 2005] [error] [client 82.112.195.101] client denied by server configuration: /home/datenrou/public_html/blog/index.php
[Wed Aug 17 11:07:46 2005] [error] [client 216.86.156.205] client denied by server configuration: /home/datenrou/public_html/blog/index.php
[Wed Aug 17 11:07:46 2005] [error] [client 216.86.156.205] client denied by server configuration: /home/datenrou/public_html/blog/index.php


This seems to work, the stats say, that quite a lot of requests are 403'd. Goood thing.
Quote:

#reqs: status code
-----: -----------
61238: 200 OK
38241: 403 Access forbidden
Back to top
View user's profile Send private message
Pneumatus
New Poster
New Poster

Joined: 22 Aug 2005
Posts: 8
Reputation: 15.6Reputation: 15.6 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Thu Aug 25, 2005 14:16    Post subject: Reply with quote

In addition to adding BlockUntrustedVisitors(); to b2evocore/_blog_main.php I have (and would reccomend) you add the same to htsrv/comment_post.php and htsrv/trackback.php to nerf the spammers that direct-access these files to post comments and trackbacks rather than going through the proper URLs.

Currently still in testing on my blog but I don't think i've seen a DNS Blacklisted IP manage to post a comment or a trackback so far.
Back to top
View user's profile Send private message
mweis
New Poster
New Poster

Joined: 27 Jul 2005
Posts: 5
Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5Reputation: 9.5 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Sun Aug 28, 2005 8:27    Post subject: Reply with quote

Pneumatus wrote:

Currently still in testing on my blog but I don't think i've seen a DNS Blacklisted IP manage to post a comment or a trackback so far.


Good thing, but have no spammer accessing these pages directly. These are the most wanted in the blog:

    29989: 32.00%: Aug/27/05 9:57 PM: /blog/index.php
    11513: 9.25%: Aug/26/05 4:50 PM: /blog/index.php?blog=1&disp=stats
    6122: 3.71%: Aug/26/05 4:50 PM: /blog/index.php?blog=5&disp=stats
    1392: 2.46%: Aug/27/05 9:53 PM: /blog/index.php?blog=1_
    1315: 0.02%: Aug/23/05 1:28 AM: /blog/index.php?blog=4&disp=stats
Back to top
View user's profile Send private message
kwa
Hooked :)
Hooked :)

Joined: 21 Jan 2005
Posts: 261
Reputation: 33.2Reputation: 33.2Reputation: 33.2 add or subtract from this member's reputationadd or subtract from this member's reputation
votes: 1

PostPosted: Wed Sep 07, 2005 15:07    Post subject: Reply with quote

A b2evolution blogger has implemented the BlockUntrustedVisitors() hack and he notices a reduction of 50% of his CPU usage (English translation here).

The following image shows his server CPU usage among time:

Who's going to find first at what time he's implemented the BlockUntrustedVisitors() hack?

Now, since I noticed about 30% of my blogs requests lead to redirects, mainly made by .htaccess antispam filtering and BlockUntrustedVisitors() filtering, I wonder if it isn't more interesting to first call the cheap BlockUntrustedVisitors() function, then filter using a local blacklist (and thus stopping using .htaccess to filter spam).

_________________
Les perles du chat | Une araignée au plafond | New Development Blog Coming Soon...
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Dan
New Poster
New Poster

Joined: 16 May 2005
Posts: 11
Reputation: 12 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Wed Sep 07, 2005 18:12    Post subject: Other ways to skin the cat Reply with quote

BlockUntrustedVisitors looks great, I'll have to give it a try.

I've seen a lot of talk about the stats page, so I'll point to a discussion and solution on another thread about this -- the stats discussion begins here.


I have also implemented a method to ban any IP address that repeatedly calls any script in the htsrv directory that generates a server error (e.g., 404 Not Found). I get lots of those because I regularly rename the htsrv directory -- see this thread for details.

Here's my post about this solution, and here's a link to the latest code. As of this post, the code was as follows:
PHP:
<?php
function checkBan ($errIP) {
    
$fBad dirname(__FILE__).'/badguys.txt';
    
$fBanned dirname(__FILE__).'/bannedguys.txt';
    
$htaccess dirname(__FILE__).'/.htaccess';
    
$admin_email rtrim(file_get_contents(dirname(__FILE__).'/admin_email.txt'));

    
file_exists($fBad) && $bad file($fBad);
    
file_exists($fBanned) && $banned file($fBanned);

    if (
is_array($banned) && in_array("$errIP\n"$banned)) {
        echo 
''# How did this guy get through?
    
} elseif (is_array($bad) && in_array("$errIP\n"$bad)) {
            
# Two strikes and you're out!
            # Add the bad IP to the banned list
        
$fp fopen($fBanned'a');
        
fwrite($fp"$errIP\n");
        
fclose($fp);
            
# Add the bad IP to .htaccess
        
$fp fopen($htaccess'a');
        
fwrite($fp"Deny from $errIP\n");
        
fclose($fp);
            
# Notify me of this action by email
        
$msg wordwrap("$errIP was added to the list of banned IP addresses."70);
        
$subj "New banned IP address";
        
mail($admin_email$subj$msg);
    } else {
            
# Add the IP to the bad list
            # If he does it again, he'll get banned
        
$fp fopen($fBad'a');
        
fwrite($fp"$errIP\n");
        
fclose($fp);
    }
# end sub
?>


N.B.: On my site, PHP scripts run with my privileges, not the privileges of the webserver (e.g., nobody or apache). If that's not your situation, any files that checkban writes to will need to be world writeable. Obviously, that's not a good idea for .htaccess, so you'll want to come up with a secure alternative.

Also, if you get a lot of traffic (which I don't), you'll probably want to comment out the line that sends an email notification every time an IP is blocked. I like that, though, because I want to know that my script is working.
Back to top
View user's profile Send private message Visit poster's website
kwa
Hooked :)
Hooked :)

Joined: 21 Jan 2005
Posts: 261
Reputation: 33.2Reputation: 33.2Reputation: 33.2 add or subtract from this member's reputationadd or subtract from this member's reputation
votes: 1

PostPosted: Mon Sep 12, 2005 20:48    Post subject: Reply with quote

For all users having implemented the BlockUntrustedVisitors(): the www.empty.us site checked by the BlockUntrustedVisitors() function appears to be down making accessing from external URLs very long (until a timeout makes the PHP going on...) I recommand to remove using that site until the next release of BlockUntrustedVisitors() that would not hang if a site is not responding.

Edit the following code in the BlockUntrustedVisitors() function:
PHP:
    $DnsblServices = array( 
      array( 
'query' => "$d.$c.$b.$a.list.dsbl.org",     'url' => 'http://dsbl.org/listing?'.$VisitorIP ),   // See http://www.dsbl.org 
      
array( 'query' => "$d.$c.$b.$a.opm.blitzed.org",   'url' => 'http://opm.blitzed.org/'.$VisitorIP ), // See http://opm.blitzed.org/info 
      
array( 'query' => "$d.$c.$b.$a.bsb.empty.us",      'url' => 'http://bsb.empty.us/lookup/?ip='.$VisitorIP ), // See http://bsb.empty.us/ 

      // Make sure the following is the last entry of the table. 
      // If you have to insert other entries, insert them above that comment. 
      
array( 'query' => $ReferrerDomain.'.bsb.empty.us''url' => 'http://bsb.empty.us/lookup/?h='.$ReferrerDomain // See http://bsb.empty.us/ 
and remove the two last entries, so you should see:
PHP:
    $DnsblServices = array( 
      array( 
'query' => "$d.$c.$b.$a.list.dsbl.org",     'url' => 'http://dsbl.org/listing?'.$VisitorIP ),   // See http://www.dsbl.org 
      
array( 'query' => "$d.$c.$b.$a.opm.blitzed.org",   'url' => 'http://opm.blitzed.org/'.$VisitorIP ), // See http://opm.blitzed.org/info 

The next version (to be released) won't wait long before skipping a query of a down site.

I am very interested in any ways to filter spam, especially blog-specific spam (mainly referrers, comments and trackbacks). If you know any, you're welcome to share your ideas on the Request for The Ultimate Antispam Plug-in or Hack thread.

_________________
Les perles du chat | Une araignée au plafond | New Development Blog Coming Soon...
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
guchuj05
New Poster
New Poster

Joined: 02 Dec 2005
Posts: 12
Reputation: 5.4Reputation: 5.4Reputation: 5.4Reputation: 5.4Reputation: 5.4 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Sun Feb 12, 2006 15:08    Post subject: Blocking referrers Reply with quote

kwa wrote:


How to implement it?
If you haven't created a conf/hacks.php file before, create a dummy (empty) conf/hacks.php file with:
Code:
<?php

/* PHP code to be inserted here */

?>




What do you do if you already have a conf/hacks.php file that does something else?
Thanks
J
Back to top
View user's profile Send private message
whoo
Boss Lady
Boss Lady

Joined: 25 Dec 2004
Posts: 1293
Reputation: 100.1 add or subtract from this member's reputationadd or subtract from this member's reputation
votes: 19

PostPosted: Sun Feb 12, 2006 15:45    Post subject: Reply with quote

add whatever code your trying to use below or above whatever already exists in the file.. for instance:

PHP:
<?php
function some-function() {.......
.....
}
function 
some-other-function() {
.....
}
?>


or you can just enclose them sperately, like so:

PHP:
<?php
function some-function() {.......
.....
}
?>

<?php
function some-other-function() {
.....
}
?>

_________________

Do you use last.fm?
Try out my nifty badge generator!
Back to top
View user's profile Send private message Visit poster's website
chrpeder
New Poster
New Poster

Joined: 28 May 2004
Posts: 30
Reputation: 31.3Reputation: 31.3Reputation: 31.3 add or subtract from this member's reputationadd or subtract from this member's reputation

PostPosted: Thu Apr 06, 2006 17:54    Post subject: Reply with quote

when I installed the BlockUntrustedVisitors hack it was not possible to access my blog from google searchs.

I got redirected to the dsbl.org website when pressing a search result for my blog on google.

DSBL: Listing Data
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    b2evolution Forum Index -> Fighting spam! All times are GMT - 5 Hours
Goto page Previous  1, 2
Page 2 of 2


 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
b2evolution Support Forum RSS Feed Forums powered by php Bulletin Board