Editing Infrastructure Machines (section)

=== Install OS ===

Install FreeBSD 8.3 amd64

* partition map:
<pre>/ 500m
swap 4096m
/var 256m
/tmp 256m
/usr ~</pre>

* edit /etc/make.conf 
Castle:
<pre>echo "WITHOUT_X11=yes \
KERNCONF=bwdb \
BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf</pre>

i2b:
<pre>echo "WITHOUT_X11=yes \
KERNCONF=bwdb2 \
BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf</pre>

* add settings to /boot/loader.conf and /boot.config

<pre>echo "-Dh" >> /boot.config

echo 'console="comconsole,vidconsole" \
boot_multicons="YES" \
boot_serial="YES" \
comconsole_speed="115200"' >> /boot/loader.conf</pre>

* turn off all ttyv's except 0 and 1 in /etc/ttys
also turn on ttyu0, change type to vt100:
<pre>vi /etc/ttys

ttyv2   "/usr/libexec/getty Pc"         cons25  off secure
ttyv3   "/usr/libexec/getty Pc"         cons25  off secure
ttyv4   "/usr/libexec/getty Pc"         cons25  off secure
ttyv5   "/usr/libexec/getty Pc"         cons25  off secure
ttyv6   "/usr/libexec/getty Pc"         cons25  off secure
ttyv7   "/usr/libexec/getty Pc"         cons25  off secure
# Serial terminals
# The 'dialup' keyword identifies dialin lines to login, fingerd etc.
ttyu0   "/usr/libexec/getty std.9600"   vt100   on secure

kill -1 1</pre>

on console server:
 vi /etc/remote
(rename port to jail8 depending on where and which digi plugged into)
test serial console


* populate hosts
i2b:
<pre>echo "69.55.230.10 backup2" >> /etc/hosts
echo "69.55.230.11 backup1" >> /etc/hosts
echo "10.1.2.3 backup3" >> /etc/hosts</pre>

castle:
<pre>echo "10.1.4.3 backup2 backup2.johncompanies.com" >> /etc/hosts
echo "10.1.4.8 backup1 backup1.johncompanies.com" >> /etc/hosts
echo "10.1.4.4 mail mail.johncompanies.com" >> /etc/hosts
</pre>

* put key in authorized_keys on backup1 and  backup2
 cd
 ssh-keygen -t dsa -b 1024 
(default location, leave password blank)

castle:
 cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys' 
 cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' 

i2b:
 cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys' 
 cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' 
 cat /root/.ssh/id_dsa.pub | ssh backup3 'cat - >> /root/.ssh/authorized_keys' 

confirm that you can ssh to backup2 and backup1 (and backup3 if at i2b) without getting a login prompt

 ssh backup1 hostname
 ssh backup2 hostname

* edit root's path and login script:
 vi /root/.cshrc

Change alias entries (add G):
<pre>alias la        ls -aG
alias lf        ls -FAG
alias ll        ls -lAG
alias ls        ls -AG
</pre>

and alter the prompt, set the following:
 set prompt = "`/bin/hostname -s` %/# "

* install cvsup
 cd /usr/ports/net/cvsup-without-gui 
 make install clean; rehash; mail -s 'cvs installed' support@johncompanies.com < /dev/null

* get latest sources for this release:
<pre>cd /usr/src 
echo "*default host=cvsup4.freebsd.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8_3\
*default delete use-rel-suffix\
*default compress\
src-all" > sup

cvsup sup ; mail -s 'cvs sup done' support@johncompanies.com < /dev/null</pre>

* configure new kernel

 cd /usr/src/sys/amd64/conf 
 scp backup2:/mnt/data4/build/freebsd/kern_config-bwdb-8.3-amd64 ./bwdb

Edit config and change name:
 vi bwdb
 ident  bwdb

* build, install kernel and world

<pre>cd /boot

mv kernel kernel.GENERIC
cd kernel.GENERIC
cd /usr/src
make buildkernel installkernel

make buildworld ; mail -s 'buildworld done' support@johncompanies.com < /dev/null
(2450: 1:56min, supermicro: 59mins, 2950: 38mins)
make installworld 
(2450: 3min, supermicro: 1min, 2950: :34)
mergemaster -i</pre>

* populate /etc/rc.conf with IPs and NFS settings

castle:
<pre>vi /etc/rc.conf

hostname="bwdb.johncompanies.com"
kern_securelevel_enable="NO"
portmap_enable="NO"
sendmail_enable="NO"
usbd_enable="YES"

xntpd_enable="YES"
nfs_client_enable="YES"
nfs_reserved_port_only="YES"
ifconfig_fxp0="inet 10.1.4.203 netmask 255.255.255.0"
ifconfig_em0="up promisc"
defaultrouter="10.1.4.1"
snmpd_enable="YES"

inetd_enable="YES"
inetd_flags="-wW -a 10.1.4.203"
fsck_y_enable="YES"
background_fsck="NO"
sshd_enable="YES"
ipfw_load="YES"</pre>

i2b:
<pre>vi /etc/rc.conf

hostname="bwdb2.johncompanies.com"
kern_securelevel_enable="NO"
portmap_enable="NO"
sendmail_enable="NO"
usbd_enable="YES"

xntpd_enable="YES"
nfs_client_enable="YES"
nfs_reserved_port_only="YES"
ifconfig_fxp0="inet 10.1.2.4 netmask 255.255.255.0"
ifconfig_em0="up promisc"
defaultrouter="10.1.2.1"
snmpd_enable="YES"

inetd_enable="YES"
inetd_flags="-wW -a 10.1.2.4"
fsck_y_enable="YES"
background_fsck="NO"
sshd_enable="YES"
ipfw_load="YES"</pre>

* reboot. Confirm new kernel is loaded

 uname -a

* update ports:
<pre>cd /usr/ports
echo "*default host=cvsup4.FreeBSD.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8_3\
*default delete use-rel-suffix\
*default compress\
ports-all tag=." > sup

cvsup sup; mail -s 'cvs sup ports done' support@johncompanies.com < /dev/null</pre>

* Install raid mgmt tool

<pre>cd /usr/local/sbin
scp backup2:/d4/build/3ware/tw_cli-freebsd-x86_64-9.5.0.1.tgz .
tar xzf tw_cli-freebsd-x86_64-9.5.0.1.tgz
rm tw_cli-freebsd-x86_64-9.5.0.1.tgz
chmod 0700 tw_cli</pre>

Test:
 ./tw_cli info c0

Grab raid check script:
 scp backup1:/usr/local/sbin/3wraidchk /usr/local/etc

Setup cronjob:
<pre>crontab -e
*/5 * * * * /usr/local/etc/3wraidchk</pre>

* install rsync from ports
 cd /usr/ports/net/rsync
 make install clean

choose default options

* install perl from ports
 cd /usr/ports/lang/perl5.8
 make install clean

choose default options

* install bb client

Compiling from source on AMD64 will not work. So, we use a linux-compiled version and rely on linux compat. Linux compat won't install on 8.x - libtool 2.4 need. So, instead we copy(ed) over linux: 
 rsync -aSHv --exclude=proc --exclude=sys 10.1.4.108:/usr/compat/linux/ /usr/compat/linux/

 adduser

Output/response:

<pre>Username: bb
Full name: bb
Uid (Leave empty for default): 1984
Login group [bb]:
Login group is bb. Invite bb into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: 
Home directory [/home/bb]:
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]: yes
Lock out the account after creation? [no]:
Username   : bb
Password   : <random>
Full Name  : bb
Uid        : 1984
Class      :
Groups     : bb
Home       : /home/bb
Shell      : /bin/sh
Locked     : no
OK? (yes/no): yes</pre>

 cd /usr/home/bb
 scp backup2:/mnt/data4/build/bb/bb-freebsd_linuxcompat.tgz .
 tar xzf bb-freebsd_linuxcompat.tgz

edit /home/bb/bbc1.9e-btf/etc/bb-hosts with something like:

 echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \
 10.1.4.203 bwdb.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts

Edit for machine name and private IP.

if this machine is at i2b:
 echo "69.55.230.2 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \
 10.1.2.4 bwdb2.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts

<pre>vi /home/bb/bbc1.9e-btf/ext/openfiles 

MACHINE="bwdb,johncompanies,com"      # HAS TO BE IN A,B,C FORM</pre>
Edit for machine name.

Have bb watch for flow-capture, mysql
<pre>cat >> /home/bb/bbc1.9e-btf/etc/bb-proctab
localhost: flow-capture :
localhost: mysqld :</pre>


<pre>cd /usr/home/bb/bbc1.9e-btf/etc
./bbchkcfg.sh 
(y to questions)
./bbchkhosts.sh
(ignore ssh errors)
cd ../..
chown -R bb .
su bb
cd
cd bbc1.9e-btf
./runbb.sh start
more BBOUT 
(look for errors)
exit</pre>

Put in script to start bb @ boot:
 echo 'su - bb -c "cd /home/bb/bbc1.9e-btf; ./runbb.sh start"' > /usr/local/etc/rc.d/bb.sh
 chmod +x /usr/local/etc/rc.d/bb.sh


If this is at i2b, punch a hole in the firewall to allow it to communicate with bb monitor:

ipfw add 00096 allow tcp from 66.181.18.0/27 to 69.55.230.2


* configure bb on mail
<pre>vi /usr/home/bb/bbsrc/bb1.9e-btf/etc/bb-hosts
10.1.4.203 bwdb.johncompanies.com # ssh

su bb
cd
bbsrc/bb/runbb.sh restart ; exit</pre>

* configure ntp server
Castle:
 echo "server 10.1.4.1" > /etc/ntp.conf

I2b:
 echo "server 10.1.2.1" > /etc/ntp.conf

<pre>/usr/sbin/ntpd -p /var/run/ntpd.pid
sleep 2; ntpq -p</pre>
(confirm it’s able to reach our time server)

<pre>echo '/usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift' > /usr/local/etc/rc.d/ntp.sh
chmod 0700 /usr/local/etc/rc.d/ntp.sh</pre>

* fwd and reverse lookups on ns1c
vr johncompanies.com
 (edit the PTR too)

* setup backups
<pre>echo '#\!/bin/sh\
backupdir=/data/bwdb/current\
server=backup1\
\
## ENTRY /etc\
## ENTRY /usr/home/flowbin\
## ENTRY /usr/home/database' > /usr/local/etc/backup.config</pre>

Castle:
setup backup dirs:
 ssh backup1 mkdir -p /data/bwdb/current
on backup1, add the system to 
 vi /usr/local/sbin/snapshot_rotate

I2b:
setup backup dirs:
 ssh backup3 mkdir -p /data/bwdb/current
on backup3, add the system to 
 vi /usr/local/sbin/snapshot_archive


Copy over the backup script:
 scp backup2:/d4/bin/freebsd8.x/rsync.backup /usr/local/etc/

Edit rsync.backup and change <tt>config</tt> var to point to correct config file location: <tt>/usr/local/etc/backup.config</tt>

<pre>crontab -e
5 0 * * * /usr/local/etc/rsync.backup</pre>

* make /root/logs
 mkdir /root/logs

* edit sshd_config for security

<pre>vi /etc/ssh/sshd_config
ListenAddress 10.1.4.203
PermitRootLogin yes

kill -1 `cat /var/run/sshd.pid`</pre>

Edit for private IP.

* snmp

(Before doing this you may need to take down the firewall and also add to resolv.conf 69.43.143.41)
<pre>
cd /usr/ports/net-mgmt/net-snmp
make install clean
(defaults)

cat >> /etc/rc.conf
snmpd_enable="YES"
snmpd_flags="-a"
snmpd_conffile="/usr/local/share/snmp/snmpd.conf"
snmptrapd_enable="YES"
snmptrapd_flags="-a -p /var/run/snmptrapd.pid"

cat > /usr/local/share/snmp/snmpd.conf
rocommunity  jcread 10.1.4.5
rocommunity  jcread 10.1.4.202
</pre>