Editing NetHere (section)

=== Mail servers ===
==== Organization ====
<pre>


                                NetHere Mail Servers
                                ====================




         Customers              Inbound Mail            Outbound Mail
     /\          ||                 ||                    /\
     ||          ||                 || smtp               ||
     ||          ||                 \/                    ||
     ||          ||             ServerIron                ||
     ||          ||                 ||                    ||
     ||          ||                 ||                    ||
     ||          ||   smtp          \/                    ||
     ||          || =========>  mta-1    mta-2  ====>   relay-1  <====  unixweb-* servers
     ||          ||                 ||   /\             relay-2         ntweb servers
http ||     pop3 ||                 ||   ||
     ||     imap ||                 \/   +--------------+
     ||          ||             ServerIron              |
     ||          ||                 ||                  |
     ||          ||                 ||                  |
     ||          ||                 \/                  |
     ||          ||             nsrbl-1   nsrbl-2       |
     ||          ||         (spamhaus, dcc blacklist)   |
     ||          ||                 ||                  |
     ||          \/                 \/                  |
     \/         mailbox-1  <=== scan-1    scan-2        |
  webmail <===> mailbox-2    (sendmail/amavisd/sophos)  |
          imap  mailbox-3                               |
                 /\     /\                              |
                 ||     ||                              \/
             nfs ||     +--------------------------->  mta-db
                 ||                                    (LDAP)
                 \/
             sndg-netapp-1
             sndg-netapp-2
             sndg-netapp-3


</pre>

==== Overview ====
mta-1.mail.nethere.net- Primary servers for all inbound and outbound mail.
mta-1.mail.nethere.net-  Primary servers for all inbound and outbound mail.

nsrbl-1.nethere.net- Realtime blacklist check using dccd and spamhaus.  The mta servers query the ip address of the mail server against the blacklist before transferring the email to the scanning servers.

nsrbl-2.nethere.net
 
scan-2.mail.nethere.net- Scans email with Sophos (currently disabled) and Spam Assasin before sending it to the mailbox servers.
scan-2.mail.nethere.net Scans email with Sophos (currently disabled) and Spam Assasin before sending it to the mailbox servers.

mailbox-1.mail.nethere.net- Front end server for the mail directories.  Queries LDAP server for miscellaneous customer information including the directory the email should be stored in, passwords, spam sensitivity levels, etc. before delivering email to appropriate directory.
mailbox-2.mail.nethere.net
mailbox-3.mail.nethere.net

Mail is stored in maildir format:

/nfs/<nfs_number>/<platform>/u/s/username

sndg-netapp-1-new.nethere.net- Solaris system using zfs set up as a nfs mount on mailbox-1, -2, and -3.  Email is stored on these platforms for all customers.
sndg-netapp-2-new.nethere.net
sndg-netapp-3-new.nethere.net

mx-1.nethere.net- spooling mail server

mailx-1.nethere.net- all outbound mails originating from a web server is sent out this email server.  All web servers are set to use mailx.nethere.net which resolves (through the serveriron) to either mailx-1.nethere.net and mailx-2.nethere.net.  Since we shut down mailx-2, all outbound mail from the web servers goes through mailx-1.  This was to prevent all the customers email from being blacklisted as a result of a web site being compromised.

mta-db.mail.nethere.net- database server for the mta servers.  All postfix databases are stored here.

webmail.nethere.net- Hosts the webmail program for webmail.nethere.net, webmail.simplyweb.net and webmail.znet.net.

==== Client mail settings ====

Inbound Mail Server:

POP3
pop3.nethere.net
pop3.znet.net
pop3.simplyweb.net

IMAP
imap.nethere.net
imap.znet.net
imap.simplyweb.net

Outbound Mail server:

mail.nethere.net
mail.znet.net
mail.simplyweb.net

Webmail:

webmail.nethere.net
webmail.znet.net
webmail.simplyweb.net

Control Panel:

cp.nethere.net
cp.znet.net
cp.simplyweb.net

==== nsrbl Disk Quota Warning ====

In the event you get a nagios regarding space on the two nsrbl servers, 
it is most likely the result of the dccd databases getting too large.  
To clear out space, log into each nsrbl server and execute the following:

df -h

cd /var/dcc/libexec

./stop-dccd

cd /var/dcc

rm dcc_db*

sync ; sync

cd /var/dcc/libexec

./start-dccd

ps auxw | grep dccd

====Update outbound quota for a single ip address====

We restrict the total number of emails that a customer can send to 2000 email recepients per 24 hour period.  This is tracked by the ip address of the computer that connects to the mta server when it sends an outbound emal.  The ip address and count is stored in the postfix database server, mta-db.mail.nethere.net.  To increase the quota for a single ip address, you can log into either mta-1, mta-2, or console direcly into the database server.  To use one of the mta servers:

<pre>
1.  Log onto mta-1.mail
2.  Switch to mysql
$mysql -h 10.0.0.69 -u policyd_outbound -p
username:  policyd_outbound
password:  p0stf1x!
3.  Review database information
mysql>show databases;
4.  Switch to the policyd table. 
mysql>use policyd;
5.  Review table header information
mysql>show tables;
mysql>describe throttle;
6.  Verify the ip address is above quota
mysql>select * from throttle where _from='xxx.xxx.xxx.xxx';   #this is the ip address of the account you are increasing.
7.  Increase the maxium number of recepients the ip address can send to
mysql>update throttle SET _rcpt_max = _rcpt_max + 20000 where _from='24.249.205.66';
8.  Verify
mysql>select * from throttle where _from='xxx.xxx.xxx.xxx';
</pre>

====Remove a blocked mail server ====

The message in maillog helps identify where it is blocked.

<pre>
Relaying denied due to excessive spam                             admin-2:/dist/files/nhmta/nethere/conf/postfix/common/client_reject
Sender address rejected: Relaying denied due to Spam              admin-2:/dist/files/nhmta/nethere/conf/postfix/common/sender_reject
Relaying denied due to SPAM                                       policyd???
Client host rejected: ... listed at zen.spamhaus.org=127.0.0.2    nsrbl-1:/named/rbldns/zones/*.spamhaus.org
status=sent (250 2.7.1 Ok, discarded, id=95886-33 - SPAM)         user mail protection in provisioning????

</pre>

====Remove a mail server from policyd blacklist====

The ip address of a mail server can be placed on a blacklist for several reasons, to high of rate of inbound email from a single ip address, incorrectly formatted helo, etc.  In the event you need to remove a mail server's ip from the policyd blacklist, you can log into either mta server of connect directly to the policyd database server, mta-db.mail.nethere.net.  To remove the ip address using a mta server:

<pre>
1.  Log onto mta-1.mail
2.  Switch to mysql
$mysql -h 10.0.0.69 -u policyd_inbound -p policyd
username:  policyd_inbound
password:  p0stf1x!
3.  Display database information
mysql>show databases;
4.  Use policyd database
mysql>use policyd;
4.  Display table information
mysql>show tables;
mysql>describe blacklist;
5.  Verify ip address has been blacklisted
mysql>select * from blacklist where _blacklist='xxx.xxx.xxx.xxx'; 
6.  Delete the entry
mysql>delete from blacklist where _blacklist='xxx.xxx.xxx.xxx';
7.  Verify
mysql> select * from blacklist where _blacklist='xxx.xxx.xxx.xxx';
</pre>

The suspected mail server may also be in the helo table.

<pre>
mysql> describe helo ;
1.  Check the IP address
mysql> select * from helo where _host='xxx.xxx.xxx.xxx' ;
2.  Check the server helo name 
mysql> select * from helo where _helo like '%server%' ;
3.  Delete the entry
mysql> delete from helo where _host='xxx.xxx.xxx.xxx' ;
mysql> delete from helo where _helo like '%server%' ;
</pre>

====Remove a mail server from spamhaus blacklist====

ssh to nsrbl-1.mail  and nsrbl-2.mail

<pre>

cd /named/rbldns/zones 

find which file the IP Address is in.

grep <IP Addr> *.spamhaus.org
co -l <xxx.spamhaus.org>
vi xxx.spamhaus.org
ci -u <xxx.spamhaus.org>

</pre>

====Add ip address to client_reject====

The client_reject list is one of the ways we combat spam.  If we receive spam compaints from a mail server or otherwise identitfy a mail server as a source of spam, we can add it to a client_reject list.  An email sent from a mail server on the client_reject list is rejected with a notice saying "Relaying denied due to excessive spam". To add an ip address to the list, we use admin-2.  Once the list is updated on admin-2, it is pushed out to mta-1 and mta-2.  To add an ip address on admin-2:

<pre>
Log into admin-2.nethere.net.
bash-3.00# cd /dist/files/nhmta/nethere/conf/postfix/common
bash-3.00# co -l client_reject
RCS/client_reject,v  -->  client_reject
bash-3.00# vi client_reject

-Add the ip address to the bottom of the list in the appropriate format.  It you want to reject any email from the mail server, you would use:
xxx.xxx.xxx      REJECT Relaying denied due to excessive spam
where xxx.xxx.xxx is the /24 subnet of the mail server the spam originated from.  Do not place a period after the last octet.  For example, to add a /16 subnet you would use:
xxx.xxx          REJECT Relaying denied due to excessive spam
The REJECT key word tells postfix to reject the email.  If you want to allow email from a /24 subnet, use the following format:
xxx.xxx.xxx      OK
This tells postfix to accept email from that subnet.

bash-3.00# ci -u client_reject
RCS/client_reject,v  <--  client_reject
new revision: 1.286; previous revision: 1.285
enter log message, terminated with single '.' or end of file:
>> .
done
bash-3.00# make
/nethere/software/nhmta/postfix-2.4.5/sbin/postmap -C . client_reject
mv client_reject.db maps/hash/client_reject.db
bash-3.00# cd /dist/rdist
bash-3.00# gmake nhmta-update
updating host mta-1.mail.nethere.net
updating: /dist/files/nhmta//nethere/conf/postfix/common/maps/hash/client_reject.db
updating: /dist/files/nhmta//nethere/conf/postfix/common/client_reject
updating host mta-2.mail.nethere.net
updating: /dist/files/nhmta//nethere/conf/postfix/common/maps/hash/client_reject.db
updating: /dist/files/nhmta//nethere/conf/postfix/common/client_reject
bash-3.00#
</pre>

====Free mysql disk space on mta-db.mail.nethere.net====

In response to nagios alert "[Nagios] PROBLEM alert - mta-db.mail.nethere.net/Disk mysql is WARNING", to clear space within the mysql database:

1.  ssh to mta-db.mail.nethere.net.

2.  Log into mysql and execute the following:

[root@mta-db user]# mysql -uroot -p

Enter password:  dB@dm1N!

mysql> show databases;

mysql> use policyd;

mysql> show tables;

mysql> describe helo;

mysql> describe throttle;

mysql> describe throttle_from_instance;

mysql> select * from throttle_from_instance limit 10;

mysql> select now();

mysql> select unix_timestamp(now());

mysql> select count(*) from throttle_from_instance where _expire > 1426534697;

mysql> select max(_expire) from throttle_from_instance;

mysql> truncate table throttle_from_instance;

mysql> select count(*) from throttle_from_instance;

mysql> select * from throttle_from_instance;

mysql> show tables;

mysql> optimize table throttle_from_instance;

mysql> show databases;

mysql> use policyd;

mysql> quit

[root@mta-db user]# df -h