NAME

 SyslogScan::Daemon::SpamDetector::BlockList - maintain an IP-based blocklist of spam sources

SYNOPSIS

 plugin SyslogScan::Daemon::SpamDetector as sd_

 sd_plugin SyslogScan::Daemon::SpamDetector::BlockList
        debug           3
        track_dbi_dsn   'DBI:mysql:database=mailstats;host=localhost'
        track_dbi_user  'username'
        track_dbi_pass  'passwd'
        block_dbi_dsn   'DBI:mysql:database=iimaildb;host=blockhost'
        block_dbi_user  'username'
        block_dbi_pass  'passwd'
        minblock        0.0416666666666667      # minimum block length (days)
        maxblock        1.0     # maximum block length (days)
        blockmult       1.2     # amount to grow/shrink the block
        shrinktime      0.25    # grow the block length if spammed again in less than this (days)
        growtime        0.04    # amount of times w/o a spam to shrink block length (days)
        firstblock      10      # number of spams required to trigger first blocking
        cleanup         86400   # clean up tables, remove blocks (seconds)
        blockmemory     8       # keep history of previous blocks (days)
        spammemory      8       # keep history of spams sent (days)
        recipweight     .33333333333333  # how much does an extra recipient count towards being enough spam to block
        blockcommand    ''
        repeatblocks    50      # times blocked to get worse treatment
        repeatoffender  250     # count of spams to get worse treatment
        rpmult          4       # how much more/less time repeat offenders get to grow shrink their block time
        fastspammer     20      # number of spams sent while blocked to get penealty
        fastspammermult 1.5     # blocktime multipe for fastspammers

DESCRIPTION

Track and react to the spam noticed by SyslogScan::Daemon::SpamDetector.

Track the spam in an SQL database. Build a blocklist.

Block hosts for a bit. If they keep spamming, block them for longer periods.

We reccomend that this module not be combined with SyslogScan::Daemon::SpamDetector::SpamAssassin becuase it will end up blocking forwarded mail.

CONFIGURATION PARAMETERS

The following configuration parameters are supported:

debug

Debugging 0 (off) to 5 (verbose).

minblock

The minimum amount of time to block an IP address in days. (Default: about an hour)

maxblock

The maximum amount of time to block an IP address in days. (Default: 1)

blockmult

If a site is being re-blocked within the growtime window, multiply the block time by this amount. If a site is being re-blocked outside of the shrinktime window then divide the block time by this amount. (Default: 1.2)

growtime

If a site sends more spam and thus needs re-blocking, punish with a longer block if it's needing re-blocking before this amount of time has passed. In days. (Default: about one hour)

shrinktime

If a site sends more spam and thus needs re-blocking but it has been longer than this amount of time since it was last blocked, then give it a shorter block than it had before. In days. (Default: .25)

firstblock

Don't block a site until this many spam have been recevied. (Default: 10)

block_dbi_dsn =item block_dbi_user =item block_dbi_pass

Use this database for the IP block list. The table is small.

track_dbi_dsn =item track_dbi_user =item track_dbi_pass

Use this database for tracking incoming spam. The table is large.

cleanup

Perform cleanup of old on the database this often. Seconds. (Default: 43200 - 12 hours)

blockmemory

Remember inactive blocks for this long. Days. (Default: 4).

spammemory

Remember incoming spam for this long. Days. (Default: 10).

blockcommand

Run this command when a block is created. (No default)

recipweight

When the same message is sent to multiple recipients, count the subsequent deliveries as this much of a spam. (Default: .333333333333)

repeatblocks

When an IP address has been been blocked this man times treat it specially: instead of multiplying block times by blockmult, multiply them by rpmult.

repeatoffender

When an IP address has sent this many spams, treat it specially: instead of multiplying block times by blockmult, multiply them by rpmult.

rpmult

For espeically bad sources, instead of multiplying block times by blockmult, multiply them by rpmult. (Default: 4)

fastspammer

If this many spams are sent while being blocked (presumably because the block doesn't cover the entire network), then multiply the block time by fastspammermult.

fastspammermult

Block multiplier penealty for fast spammers. (Default: 1.5)

cleanupfirst

Clean out old entries from the database tables at startup? (Default: 1)

TABLE CREATION

SQL tables are not automatically created. Here are examples for Mysql5:

 CREATE TABLE spam_received (
        ip              VARCHAR(16),
        spamtime        DATETIME,
        messageid       TEXT,
        recipients      INT,
        host            TEXT,
        score           TEXT,
        matched         TEXT,
        INDEX           ip_index(ip)
 ) TYPE=MyISAM;

 CREATE TABLE block_history (
        ip              VARCHAR(16),
        blockstart      DATETIME,
        blockend        DATETIME,
        blockcount      INT,
        spamcount       INT,
        messageid       TEXT,
        countdays       INT,
        PRIMARY KEY     (ip),
        INDEX           done(blockend)
 ) TYPE=MyISAM;

Replace "Los Angeles" with a city in your time zone.

 drop view blocking;
 CREATE VIEW blocking AS
 SELECT ip, 
        CONCAT("450 blocked for ", 
        SUBTIME(TIMEDIFF(blockend,blockstart),
                SEC_TO_TIME(SECOND(TIMEDIFF(blockend,blockstart)))),
        " until ", ADDTIME(blockend, timediff(now(), utc_timestamp())),
        "-Los Angeles time due to ", spamcount, " spams in ", countdays, 
        " days, for example <", messageid, ">") AS message,
        SUBTIME(TIMEDIFF(blockend,blockstart),
                SEC_TO_TIME(SECOND(TIMEDIFF(blockend,blockstart)))) AS blocktime,
        case when blockend > utc_timestamp() then 1 else 0 end AS active,
        ADDTIME(blockend, timediff(now(), utc_timestamp())) as blockend_local,
        spamcount
 FROM   block_history;

USEFUL QUERIES

To watch what's going on with the block list, the following queries are helpful.

 select ip, COUNT(*), SUM(recipients) from spam_received 
 group by ip order by COUNT(*) desc limit 25;

 select COUNT(*), SUM(recipients) from spam_received;

 select active, count(*), AVG(blocktime), AVG(spamcount), MAX(spamcount)
 from blocking group by active;

 select ip, message from blocking where active = 1 order by spamcount;

 select spamcount, COUNT(*), AVG(blocktime), AVG(spamcount)
 from blocking where active = 1 group by spamcount order by spamcount;

 select blocktime, count(*), MIN(spamcount), AVG(spamcount), MAX(spamcount) 
 from blocking where active = 1 group by blocktime order by blocktime;

 select ip, blocktime, spamcount, active
 from blocking 
 order by spamcount desc
 limit 25;

 select count(*), SUM(active) from blocking where spamcount > 250;

 select truncate(spamcount,-1), COUNT(*), AVG(blocktime), AVG(spamcount)
 from blocking where active = 1 group by truncate(spamcount,-1) order by truncate(spamcount,-1);

POSTFIX CONFIG

To configure Postfix to block using this block list, include a line like:

 smtpd_client_restrictions = check_client_access mysql:/etc/postfix/mysql_spam_senders.cf

Then include a query in the referenced file:

 hosts = localhost maildb
 dbname = spamdb
 user = dbuser
 password = dbpass
 query = SELECT message FROM blocking WHERE ip = '%s' AND active = 1

If your postfix doesn't have SQL support or you're using a less capable mailer, you can dump the block list into a file:

 mysql -N -B --host=mx --database=spamdb --user=dbuser \
 -e 'select ip, message from blocking where active = 1' \
 > file

DNS BLOCKLIST

One way to turn this blocklist into a DNS-based blocklist that can be used like other blocklists is to use rbldnsd(8).

Set up a SyslogScan::Daemon::Periodic job to rebuild the blocklist:

 plugin SyslogScan::Daemon::Periodic 
        debug   1
        period  120
        command '/usr/local/bin/update_blocklist'

The update_blocklist script I use is:

 #!/bin/sh -e

 SUB=1171308796
 SECONDS=`date +%s`
 SEQN=`expr $SECONDS - $SUB`
 DIR="/var/lib/rbldns"
 ZONE="blocked"

 sed "s/SERIAL/$SEQN/" < $DIR/$ZONE.head > $DIR/$ZONE.new

 mysql -N -B --host=MYDBHOST --database=MYDB --user=MYDBUSER \
   -e 'select   ip, concat(":1:",message) \
       from     blocking where active = 1' \
   | sed 's/:450 /:/' \
   >> $DIR/$ZONE.new

 mv $DIR/$ZONE.new $DIR/$ZONE

The /var/lib/rbldns/blocked.head file I use:

 $SOA 1800 ns.idiom.com. muir.idiom.com. SERIAL 300 60 3600 300
 $NS 1800 MYHOSTNAME
 $TTL 120

I start rbldnsd(8) with:

 /usr/sbin/rbldnsd -p /var/run/rbldnsd.pid \
  -r/var/lib/rbldns -4 -bMYIPADDRESS/53 \
  spammers.MYDOMAIN:ip4set:blocked

Then, to use it with postfix(1), I set up a rbl_reply_maps so that I can give a temporary failure.

 smtpd_client_restrictions =
  permit_mynetworks,
  reject_unauth_pipelining,
  reject_rbl_client cbl.abuseat.org,
  reject_rbl_client spammers.MYDOMAIN

 rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps

My rbl_rply_maps file:

 cbl.abuseat.org   $rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason}
 spammers.MYDOMAIN 450 4.7.1 Service unavailable; $rbl_class [$rbl_what] $rbl_reason

SEE ALSO

SyslogScan::Daemon::SpamDetector

THANK THE AUTHOR

Hire the author to do some perl programming on your behalf!

LICENSE

Copyright(C) 2006 David Muir Sharnoff <muir@idiom.com>. This module may be used and distributed on the same terms as Perl itself.