Editing Infrastructure Machines (section)

= mail =
== Summary ==
This machine (mail) is the swiss army knife of the company, playing host to many services and functions. 

* Location: castle, cab 3-7
* OS: FreeBSD 4.10 x86
* Networking: Priv IP: 10.1.4.5, Pub IPs: 69.55.230.2, 69.55.225.225 (ns1c jail), 69.55.230.9. 1 onboard and 1 PCI
* Hardware: Dell 2450. 4 SCSI SCA drive bays (2 columns of 2, drive 0 top left, drive 1 just below) all hot-swap. Dual power supply. 
* Drives: two 36 GB (2 x 36GB) RAID1 arrays running on an Adaptec-based, Dell-branded (perc) RAID card.

== Services Provided ==
* mail
* web
* mysql
* bigbrother server/pager
* snmp
* named in a jail (ns1c)

== email ==
This server hosts mail for johncompanies.com (mail.johncompanies.com). Sendmail 8.13.6/8.13.6 is listening on 69.55.230.2 port 25 for incoming mail. Relaying is allowed per /etc/mail/relay-domains
Other addresses (aliases) are defined per /etc/mail/aliases

The following active users have mail hosted on this server:
* dave
* linux
* support
* payments
* sales
* tech1
* info

Traditionally, mail is checked via shell apps (pine). qpopper (pop3s) is running to allow mail downloading. Checking mail in this way causes an opened INBOX in pine to lock read-only. For this reason, we tee incoming mail to support and linux to tech1.

Procmail rules are setup to filter spam and send text messages. They are enabled for info, support, linux, tech1, dave and can be found in ~/Procmail/, for example:
<pre># more ~support/Procmail/rc.emergency
:0c # use c only if you want to forward a copy and file the original later
* ^Subject:.*\<emergency\>
* ! ^Subject:\<re\>
  {
   :0h
   FROMANDSUBJECT=|formail -XFrom: -XSubject:

   :0fwh
   | /usr/local/bin/formail -I"Subject: " -I"To: pager@johncompanies.com" ; echo $FROMANDSUBJECT ; echo

   :0
  ! -t
  }

</pre>

control: <tt>cd /etc/mail; make stop</tt> (stop), <tt>cd /etc/mail; make start</tt> (start)

The following aliases are also in place:

<pre>debian:         linux
jobs:   info
careers:        info
#reboot:         6128102202@txt.att.net
#reboot:         8582298897@vtext.com
reboot:         pager
#pager: 8582298897@vtext.com
pager:  4158718324@txt.att.net
tech1on:  "| /usr/local/sbin/tech1on.sh"
tech1off:  "| /usr/local/sbin/tech1off.sh"</pre>

To change them, edit <tt>/etc/aliases</tt> and then run <tt>newaliases</tt>

Note on tech1: this address was setup as a read-only address to be mirrored on all email coming into support and linux. We set this up so we could easily check support mail via a pop client- popping email locks out the user in pine so checking support/linux directly via pop was not an option. When checking and responding to email that comes into tech1, care should be taken to make sure it is sent as/under an address other than tech1. This is cause tech1 is not monitored by support staff as closely as email to support/linux. Further, the tech on call may not be checking tech1. Lastly, because of the nature of the copying, you will sometimes notice certain automated email/notices are received 2x in support- this is because of/related to the tech1 mirror.

To enable it (on mail, run):

<tt>~support/tech1on.sh</tt>

To disable

<tt>~support/tech1off.sh</tt>

Or via email:

<tt>tech1on@johncompanies.com
tech1off@johncompanies.com</tt>

== IP Blocking ==

<pre>
01000 deny ip from 188.92.72.5 to any
01003 deny ip from any to 122.49.31.50
01004 deny ip from 122.49.31.50 to any
01014 deny ip from 74.208.225.225 to any
01015 deny ip from any to 216.243.118.35
01016 deny ip from 216.243.118.35 to any
01017 deny ip from any to 216.243.118.36
01018 deny ip from 216.243.118.36 to any
01020 deny ip from 112.215.0.0/18 to any   2014-08-13 Blocked PT Excelcomindo Pratama (Indonesia) for fradulent credit card attempts 
01020 deny ip from 112.215.64.0/20 to any  2014-08-13 Blocked PT Excelcomindo Pratama (Indonesia) for fradulent credit card attempts 
01022 deny ip from 120.168.0.0/24 to any   2014-08-13 Blocked Indosat 3G Broadband (Indonesia) for fradulent credit card attempts
01022 deny ip from 120.175.213.0/24 to any 2014-08-13 Blocked Indosat 3G Broadband (Indonesia) for fradulent credit card attempts

</pre>

== web == 

See [[Management_System_/_Public_Website_/_Signup|Management System / Public Website / Signup]]

== mysql ==
mysql 4.1.22 is running on port 3306

* datadir: <tt>/mnt/data1/db/mysql/</tt>
* config: <tt>/etc/my.cnf</tt>
* database: <tt>jc</tt>
* control: <tt>/usr/local/etc/rc.d/mysql-server.sh stop</tt> (stop), <tt>/usr/local/etc/rc.d/mysql-server.sh start</tt> (start)

== bigbrother ==
There is a client running on mail (which monitors the services running on mail and mail itself), installed under <tt>/usr/home/bb/bbc1.9e-btf</tt><br>
And the big brother pager/server (which displays information gathered from all bb-monitored machines, including mail) is installed under <tt>/usr/home/bb/bbsrc/bb1.9i-btf</tt>

Both are running under the user <tt>bb</tt>

Refer to [[BigBrother]] for more about use.

== DNS (ns1c.johncompanies.com) ==
ns1c is a jail running on the mail server, who's IP is 69.55.225.225

It's running from <tt>/mnt/data1/ns1c-dir</tt>

See [[DNS]] for more details

== Usage and Notes ==
* always mounted to backup1 and backup2 via nfs: 
<pre>backup2:/mnt/data1 on /backup (nfs)
backup2:/mnt/data2 on /backup2 (nfs)
backup2:/mnt/data3 on /backup3 (nfs)
backup2:/mnt/data4 on /backup4 (nfs)
backup1:/data on /backup1 (nfs)
</pre>

== Cronjobs ==
 * * * * * /usr/local/www/mgmt/mrtg/mrtg.sh > /dev/null 2>&1
Gathers up data for our mrtg/load graphs

 */5 * * * * /usr/local/bin/rsync -a root@nat2:/mnt/data1/mrtg/data/ /usr/local/www/mgmt/mrtg/data/
Gathers up data from i2b servers for our mrtg/load graphs

 40 0 * * * /usr/local/bin/rsync -a root@nat2:"/mnt/data1/mrtg/*.cfg" /usr/local/www/mgmt/mrtg
Gathers up mrtg configuration (port names) from i2b switches for our mrtg/load graphs

 41 0 * * * for f in `grep -l "mnt\/data1" /usr/local/www/mgmt/mrtg/switch-p*.cfg`; do cat $f | sed s#\/mnt\/data1#\/usr\/local\/www\/mgmt# > $f.new; mv $f.new $f; done
Gathers up mrtg configuration (port names) from castle switches for our mrtg/load graphs

 1 0 1 * * cp /usr/local/www/mgmt/html/top20ip /usr/local/www/mgmt/html/top20ip_last
 1 0 1 * * cp /usr/local/www/mgmt/html/top20customers /usr/local/www/mgmt/html/top20customers_last
 2 * * * * /usr/local/www/cronjobs/top20ip.pl > /dev/null 2>&1
 15 * * * * /usr/local/www/cronjobs/top20customer.pl > /dev/null 2>&1
 1 0 1 * * rm /usr/local/www/mgmt/html/bandtrack
Archiving and generation of bandwidth statistics presented in mgmt -> Reference -> Bandwidth

 1 0 * * * /usr/local/etc/rsync.backup
Nightly backup script

 0 1 * * * /usr/local/www/mgmt/awstats/wwwroot/cgi-bin/awstats.pl -config=jcpub -update
Public web traffic stats

 15 0 * * * rm /usr/local/www/mgmt/bwgraphs/*.png
 16 0 * * * rm /usr/local/www/am/bwgraphs/*
Cleanup for graph-related temp data generated by customers using the bandwidth reports via the AM 

 10 0 1 * * /usr/local/www/cronjobs/monthly_bandwidth_report.pl
Monthly bandwidth overage report

 */3 * * * * /usr/local/www/cronjobs/bbcheck.pl
Updates mgmt with bb monitoring issues

 5 0 * * * /usr/local/www/cronjobs/shutdownreminder.pl
Emails customers reminding them of upcoming shutdown date

 7 0 * * * /usr/local/www/cronjobs/invoice_email.pl
Emails customers who have invoices and are set to auto-email (currently no customer gets these)

 8 */4 * * * /usr/local/www/cronjobs/mysqlrepchk.pl
Checking that we are properly replicating (mysql) traffic data from bwdb to backup1

 16 0 1 * * /usr/local/www/cronjobs/purge_traffic.pl
Removed old traffic data from the traffic database (running on backup1)

 */5 * * * * chmod 0700 /usr/local/www/ccard_orders/* && mv /usr/local/www/ccard_orders/* /usr/local/www/ccard_orders/done
Secure credit card data: set root-read-only 

 25 0 * * * /usr/local/www/cronjobs/biller.pl
Enters service charges in customer billing ledgers

 10 13 * * * /usr/local/www/cronjobs/pfp_batch_gather.pl
Looks for customers with balance due and active credit card on file, prepares a payflow batch

 10 14 * * * /usr/local/www/cronjobs/pfp_batch_process.pl
Tries to collect ccard funds for items in payflow batch - communicates with payflow

 15 13 * * * /usr/local/www/cronjobs/pb_batch_gather.pl
Looks for customers with balance due and active paypal billing agreement on file, prepares a paypal batch

 15 14 * * * /usr/local/www/cronjobs/pb_batch_process.pl
Tries to collect paypal funds for items in paypal batch - communicates with paypal

 0 7 * * 1 /usr/local/www/cronjobs/email_pmt_reminder.pl
Emails customers in arrears, reminding them to pay

 0 0 1 * * /usr/bin/mail -s 'archive sent mail in pine' support@johncompanies.com < /dev/null
Reminds us to archive sent mail

 0 3 * * * /usr/local/bin/rsync -a isys.e-monitoring.net:/var/mail /backup2/isys; /usr/local/bin/rsync -a isys.e-monitoring.net:/usr/home /backup2/isys
Backup data on isys

== Regular maintenance ==
*[[Routine_Maintenance#Adaptec_Controllers|Check RAID array]]

== Building a new Mail Server ==

=== Installations ===

I used FreeBSD 11.2 

The order is important especially for the Web Server.

==== Web Server ====

I used FreeBSD 11.2
  perl 5.26 
  OpenSSL 1.0.2o-freebsd
  pcre
  apache22 
  mod_perl2
  PayflowPro
  mariadb 55 server and client

Installation order is important

install perl 5.26.2 from ports

<ore>
cd /usr/ports/lang/perl5.26/
make 
[X] PERL_64BITINT  Use 64 bit integers (on i386)
[X] USE_PERL       Rewrite links in /usr/bin
(the rest unchecked
make install
</pre>

install OpenSSL 1.0.2o-freebsd

<pre>
cd /usr/ports/
make install
</pre>

install pcre

<pre>
cd /usr/ports/
make install
</pre>


install Apache22

<pre>
cd /usr/ports/distfiles
fetch http://archive.apache.org/dist/httpd/httpd-2.2.32.tar.gz


cd /usr/ports/www/apache22/tmp
fetch --no-verify-peer http://mirror.nexcess.net/apache//httpd/httpd-2.2.34.tar.gz
tar xvzf httpd-2.2.34.tar.gz
./configure --prefix=/usr/local/apache --with-ssl=/usr/local/openssl/ --enable-ssl --enable-so --with-mpm=prefork --enable-threads --enable-mods-shared='mime alias setenvif dir' --enable-modules='mime alias setenvif dir' --with-pcre=/usr/local
make install
apachectl restart

cd /usr/ports/www/apache22
echo "DEFAULT_VERSIONS+=apache=2.2" >> /etc/make.conf
make DISABLE_VULNERABILITIES=yes
make install
</pre>


install mod_perl2

<pre>
cd /usr/ports/www/mod_perl2
echo "DEFAULT_VERSIONS+=apache=2.2" >> /etc/make.conf
make DISABLE_VULNERABILITIES=yes
make install
</pre>


install mariadb

<pre>
cd /usr/ports/databases/mariadb-103-server
echo "DEFAULT_VERSIONS+=apache=2.2" >> /etc/make.conf
make DISABLE_VULNERABILITIES=yes
make install


</pre>

==== Mail Server ====

I used Postfix for email

==== DNS Server (ns1c.johncompanies.com) ====