Editing Infrastructure Machines (section)

== build ==

* partition map:
<pre>/ 512m
swap 1G
/var 256m
/tmp 256m
/usr 5g
/mnt/data1 ~</pre>

* edit /etc/make.conf 
<pre>echo "WITHOUT_X11=yes \
KERNCONF=nat2 \
BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf</pre>

* add settings to /boot/loader.conf and /boot.config

<pre>echo "-Dh" >> /boot.config

echo 'console="comconsole,vidconsole" \
boot_multicons="YES" \
boot_serial="YES" \
comconsole_speed="115200"' >> /boot/loader.conf</pre>

* turn off all ttyv's except 0 and 1 in /etc/ttys
also turn on ttyd0, change type to vt100:
<pre>vi /etc/ttys

ttyv2   "/usr/libexec/getty Pc"         cons25  off secure
ttyv3   "/usr/libexec/getty Pc"         cons25  off secure
ttyv4   "/usr/libexec/getty Pc"         cons25  off secure
ttyv5   "/usr/libexec/getty Pc"         cons25  off secure
ttyv6   "/usr/libexec/getty Pc"         cons25  off secure
ttyv7   "/usr/libexec/getty Pc"         cons25  off secure
# Serial terminals
# The 'dialup' keyword identifies dialin lines to login, fingerd etc.
ttyd0   "/usr/libexec/getty std.9600"   vt100   on  secure

kill -1 1</pre>

on console server:
 vi /etc/remote
(rename port to jail8 depending on where and which digi plugged into)
test serial console


* populate hosts
<pre>echo "69.55.230.10 backup2" >> /etc/hosts
echo "69.55.230.11 backup1" >> /etc/hosts
echo "10.1.2.4 bwdb2" >> /etc/hosts
echo "10.1.2.3 backup3" >> /etc/hosts</pre>

* put key in authorized_keys on backup3
<pre>cd
ssh-keygen -t dsa -b 1024</pre> 
(default location, leave password blank)

<pre>cat /root/.ssh/id_dsa.pub | ssh backup3 'cat - >> /root/.ssh/authorized_keys' 
cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' 
cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys'</pre> 

confirm that you can ssh to backup3 and backup 2 without getting a login prompt

 ssh backup3 hostname

 ssh backup2 hostname

 ssh backup1 hostname


* edit root's path and login script:
 vi /root/.cshrc

Change alias entries (add G):
<pre>alias la        ls -aG
alias lf        ls -FAG
alias ll        ls -lAG
alias ls        ls -AG
alias mbm       mb mount
alias mbu       mb umount</pre>

and alter the prompt, set the following:
 set prompt = "`/bin/hostname -s` %/# "

* install cvsup
<pre>cd /usr/ports/net/cvsup-without-gui 
make install clean; rehash; mail -s 'cvs installed' dave.boodman@vtext.com < /dev/null</pre>

* get latest sources for this release:
<pre>cd /usr/src 
echo "*default host=cvsup4.freebsd.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_6_4\
*default delete use-rel-suffix\
*default compress\
src-all" > sup

cvsup sup ; mail -s 'cvs sup done' dave.boodman@vtext.com < /dev/null</pre>


* configure new kernel. 

 cd /usr/src/sys/i386/conf 
 scp backup2:/mnt/data4/build/freebsd/nat2-6.4 ./nat2

* build, install kernel and world

<pre>cd /boot

mv kernel kernel.GENERIC
cd kernel.GENERIC
cd /usr/src
make buildkernel installkernel

make buildworld ; mail -s 'buildworld done' support@johncompanies.com < /dev/null
make installworld 
mergemaster -i
</pre>

* populate /etc/rc.conf with IPs and NFS settings
<pre>vi /etc/rc.conf

hostname="nat2.johncompanies.com"
kern_securelevel_enable="NO"
portmap_enable="NO"
sendmail_enable="NO"
usbd_enable="YES"

xntpd_enable="YES"
xntpd_flags="-A -p /var/run/ntpd.pid"

nfs_client_enable="YES"
nfs_reserved_port_only="YES"
ifconfig_em0="inet 10.1.6.50 netmask 255.255.255.0"
#ifconfig_em0="inet 69.55.229.2 netmask 255.255.255.0"
#ifconfig_em0_alias0="inet 69.55.229.229 netmask 255.255.255.255"
ifconfig_fxp0="inet 69.55.229.2 netmask 255.255.255.0"
ifconfig_fxp0_alias0="inet 69.55.229.3 netmask 255.255.255.255"
ifconfig_fxp1="inet 10.1.2.1 netmask 255.255.255.0"
defaultrouter="10.1.6.1"
#defaultrouter=" 66.181.14.250"
snmpd_enable="YES"
ipnat_enable="YES"
ipnat_rules="/etc/ipnat.rules"
gateway_enable="YES"

inetd_enable="YES"
inetd_flags="-wW -a 10.1.2.1"
fsck_y_enable="YES"
background_fsck="NO"
sshd_enable="YES"</pre>

* reboot. Confirm new kernel is loaded

 uname -a

* update ports:
<pre>cd /usr/ports
echo "*default host=cvsup4.FreeBSD.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_6_4\
*default delete use-rel-suffix\
*default compress\
ports-all tag=." > sup

cvsup sup; mail -s 'cvs sup ports done' dave.boodman@vtext.com < /dev/null</pre>

* Install raid mgmt tool

<pre>cd /usr/local/sbin
fetch http://3ware.com/download/Escalade9690SA-Series/9.5.3/tw_cli-freebsd-x86-9.5.3.tgz
tar xzf tw_cli-freebsd-x86-9.5.3.tgz
rm tw_cli-freebsd-x86-9.5.3.tgz
chmod 0700 tw_cli</pre>

Test:
 ./tw_cli info c0

* install rsync from ports
<pre>cd /usr/ports/net/rsync
make install clean</pre>
choose default options

* install perl from ports
<pre>cd /usr/ports/lang/perl5.8
make install clean</pre>

* install screen from ports
<pre>cd /usr/ports/sysutils/screen
make install clean</pre>

* install bb client
<pre>adduser
Username: bb
Full name: bb
Uid (Leave empty for default): 1984
Login group [bb]:
Login group is bb. Invite bb into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: 
Home directory [/home/bb]:
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]: yes
Lock out the account after creation? [no]:
Username   : bb
Password   : <random>
Full Name  : bb
Uid        : 1984
Class      :
Groups     : bb
Home       : /home/bb
Shell      : /bin/sh
Locked     : no
OK? (yes/no): yes

cd /usr/home/bb
scp backup2:/mnt/data4/build/bb/bb-freebsd.tar .
tar xvf bb-freebsd.tar</pre>
edit /home/bb/bbc1.9e-btf/etc/bb-hosts with something like:
<pre>echo "69.55.230.2 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \
10.1.2.1 nat2.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts
vi /home/bb/bbc1.9e-btf/ext/openfiles 
MACHINE="nat2,johncompanies,com"      # HAS TO BE IN A,B,C FORM

cd /usr/home/bb/bbc1.9e-btf/etc
./bbchkcfg.sh 
(y to questions)
./bbchkhosts.sh
(ignore ssh errors)
cd ../..
chown -R bb .
su bb
cd
cd bbc1.9e-btf/src
make; make install
cd ..

vi /usr/home/bb/bbc1.9e-btf/bin/bb-cpu.sh
        $1 $TOPARGS > $BBTMP/TOP.$$
#        /usr/local/jail/bin/jtop > $BBTMP/TOP.$$

./runbb.sh start
more BBOUT 
(look for errors)
exit

echo 'su - bb -c "cd /home/bb/bbc1.9e-btf; ./runbb.sh start"' > /usr/local/etc/rc.d/bb.sh
chmod +x /usr/local/etc/rc.d/bb.sh
</pre>

Punch a hole in the firewall to allow it to communicate with bb monitor (probably already exists):

 ipfw add 96 allow ip from 66.181.18.0/27 to 69.55.230.2

* configure bb on mail:
<pre>vi /usr/home/bb/bbsrc/bb1.9e-btf/etc/bb-hosts
64.163.14.48 nat2.johncompanies.com # ssh

su bb
cd
bbsrc/bb/runbb.sh restart ; exit</pre>

* configure ntp
<pre>echo "server 69.55.230.2
server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org
server 3.pool.ntp.org
server 66.187.233.4
server 217.204.76.170
server 64.112.189.11
server 66.69.112.130
server 80.85.129.25
server 80.237.234.15
server 130.60.7.44
server 134.99.176.3
server 198.144.202.250
server 202.74.170.194
server 204.17.42.199
server 204.87.183.6
server 213.15.3.1
server 213.239.178.33
server 217.114.97.97
server 69.55.230.2" > /etc/ntp.conf</pre>

<pre>/usr/sbin/ntpd -A -p /var/run/ntpd.pid 
sleep 2; ntpq -p</pre>
(confirm it’s able to reach our time server)

 echo '/usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift' > /usr/local/etc/rc.d/ntp.sh
 chmod 0700 /usr/local/etc/rc.d/ntp.sh

* fwd and reverse lookups on ns1c
 vr johncompanies.com
 (edit the PTR too)

* setup backups, nfs mount

<pre>mkdir /backup3
echo 'backup3:/data           /backup3        nfs     rw,bg           0       0' >> /etc/fstab

echo '#\!/bin/sh\
backupdir=/data/nat2/current\
\
## ENTRY /etc ' > /usr/local/etc/backup.config</pre>

on backup3:
setup backup dirs:
 ssh backup3 mkdir -p /data/nat2/current

on backup3, add the system to 
 vi /usr/local/sbin/snapshot_archive

<pre>scp 69.55.230.2:/usr/local/etc/rsync.backup /usr/local/etc/rsync.backup 
vi /usr/local/etc/rsync.backup 
backup1 > backup3</pre>

 crontab -e
 1 0 * * * /usr/local/etc/rsync.backup

* edit sshd_config for security
<pre>vi /etc/ssh/sshd_config
ListenAddress 66.181.18.1
ListenAddress 69.55.229.2
ListenAddress 10.1.2.1

kill -1 `cat /var/run/sshd.pid`</pre>

* raid chk

<pre>cat > /usr/local/sbin/lsiraidchk
#!/usr/bin/perl

my @out = split "\n", `megarc -ldInfo -a0 -Lall|grep Status:`;

foreach (@out) {
    if ($_ =~ /DEGRADED/) { $date = `date`; chomp $date; `echo "$date: RAID ARRAY DEGRADED" >> /var/log/messages`; }
#print $_;
}</pre>

* netflow stuff
add crontab entries
<pre>crontab -e
30 3 * * * /usr/local/etc/rsync.backup
0 0 1 * * /sbin/ipfw zero
0 0 1 * * /sbin/ipfw del 3 4 5
59 23 30 * * /sbin/ipfw show > /tmp/ipfw_count
3 0 30 * * /sbin/ipfw show > /tmp/ipfw_count
0 3 * * * /etc/makefwrules.pl; /etc/makepiperules.pl; 
*/5 * * * * /usr/local/sbin/lsiraidchk

#10 0 * * * rm /var/spool/clientmqueue/*</pre>

<pre>scp /etc/makefwrules.pl user@64.163.14.48:~
scp /etc/makepiperules.pl user@64.163.14.48:~
mv /home/user/makefwrules.pl /etc
mv /home/user/makepiperules.pl /etc
touch /etc/firewall.sh
mkdir /etc/oldrules/</pre>

other binaries
	
<pre>scp /usr/local/bin/rulemaker user@64.163.14.48:~
mv ~user/rulemaker /usr/local/sbin
scp ~user/Sendmail.pm user@64.163.14.48:~
scp ~user/doswatch.pl user@64.163.14.48:~</pre>

* add nat rules
<pre>vi /etc/ipnat.rules
# sample entry
bimap fxp0 10.1.6.70/32 -> 10.1.6.59/32
#bimap fxp0 10.1.4.208/32 -> 69.55.233.196/32

ipnat -C -f /etc/ipnat.rules</pre>

* shell for user
<pre>cp /root/.cshrc ~user/
vi ~user/</pre>
change # to $

* mrtg

<pre>cd /usr/ports/net-mgmt/mrtg
make install clean</pre>
(no FONTCONFIG, v3)

this didn't work cause of libtool incompat

so manually moved files:

<pre>scp /usr/local/bin/cfgmaker user@nat2:/usr/local/bin/cfgmaker
scp /usr/local/lib/perl5/site_perl/5.6.1/MRTG_lib.pm root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/
scp /usr/local/lib/perl5/site_perl/5.6.1/SNMP_util.pm root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/
scp /usr/local/lib/perl5/site_perl/5.6.1/BER.pm root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/
scp /usr/local/lib/perl5/site_perl/5.6.1/SNMP_Session.pm root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/
scp /usr/local/bin/mrtg root@nat2:/usr/local/bin/mrtg
scp /usr/local/lib/perl5/site_perl/5.6.1/locales_mrtg.pm root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/locales_mrtg.pm
scp /usr/local/bin/rrdtool root@nat2:/usr/local/bin/rrdtool
scp /usr/local/lib/perl5/site_perl/5.6.1/mach/RRDs.pm root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/mach/RRDs.pm
rsync -av /usr/local/lib/perl5/site_perl/5.6.1/mach/auto/RRDs/ root@nat2:/usr/local/lib/perl5/site_perl/5.8.9/mach/auto/RRDs/
scp /usr/lib/libz.so.2 root@nat2:/usr/lib/libz.so.2
scp /usr/lib/libm.so.2 root@nat2:/usr/lib/libm.so.2
rsync -av /usr/local/lib/librrd* root@nat2:/usr/local/lib/
scp /usr/lib/libc.so.4 root@nat2:/usr/lib/libc.so.4

rsync -av /usr/ports/net/rrdtool root@nat2:/usr/ports/net
cd /usr/ports/net/rrdtool
make install

mkdir -p /mnt/data1/mrtg/data
scp /usr/local/www/mgmt/mrtg/template.pl root@nat2:/mnt/data1/mrtg/
scp /usr/local/www/mgmt/mrtg/host.pl root@nat2:/mnt/data1/mrtg/

cfgmaker --if-template=template.pl --show-op-down --global "options[_]: growright,bits" --global 'WorkDir: /mnt/data1/mrtg/data' --global 'Interval: 1' --global 'LogFormat: rrdtool' --global 'PathAdd: /usr/local/bin' --global 'LibAdd: /usr/local/lib' --host-template=host.pl jc292401@10.1.2.50 --output=switch-p20.cfg

cat > /mnt/data1/mrtg/mrtg.sh
#!/bin/sh
/usr/local/bin/mrtg /mnt/data1/mrtg/switch-p20.cfg

chmod 0700 /mnt/data1/mrtg/mrtg.sh

crontab -e
* * * * * /mnt/data1/mrtg/mrtg.sh 2>&1 > /dev/null</pre>

* snmp firewall block
<pre>cat > /usr/local/etc/rc.d/boot.sh
ipfw add 10 allow udp from 69.55.230.2 to any 161
ipfw add 10 allow udp from 10.1.2.1 to any 161
ipfw add 11 deny udp from any to any 161
chmod 0700 /usr/local/etc/rc.d/boot.sh</pre>