Editing Jail Server Install (section)

= FreeBSD 8.x =

== Assumptions ==

Setup instructions below assume this is DELL 2950 with an LSI-based SAS RAID card.

Server is at castle, connected to pub, private, serial and DRAC

Assuming OS loading done via IPKVM with ISO mounted via USB

Assumes at 4 drives, 2 mirrors

== Configure server BIOS ==

setup console redirect, speed 115200

set LCD string to name of server "jail8"

set date to GMT

go into RAID bios and setup mirrors

configure DRAC: TODO

== Install OS (sysinstall) ==

boot to bootonly disk for AMD version of FreeBSD, i.e. <tt>FreeBSD-8.3-RELEASE-amd64-bootonly.iso</tt>

when the install menu appears, choose <tt>custom install</tt>

=== partition menu ===
move cursor to mfid0, hit space (takes you to partition map screen). If there is only 1 mirror, there will be no option to select a specifit drive: mfid0 will be selected.

type 'a' to use entire disk<br>
type 'q' to quit and save<br>
choose 'freebsd standard mbr'

space to '''un'''select mfid0<br>
cursor down to mfid1<br>
hit space

type 'a' to use entire disk<br>
type 'q' to quit and save<br>
choose 'none' for boot mgr (leave untouched)

cursor over mfid0<br>
space<br>
(takes you into part. Screen again) 'q' to exit<br>
none for boot mgr<br>

Make sure both drives (mfid0 and mfid1) are checked and tab to ok

=== Label menu ===

Make sure mfid0 is highlighted at the top of the screen, setup the following partitions

/ 512M<br>
swap 6G<br>
/var 256M <br>
/tmp 256M<br>
/usr 5G<br>
/mnt/data1 remaining space 

All partitions except / should be setup for soft updates. If not, type 's' to enable for soft updates on all except for / (should look like UFS2+S Y under the Newfs column)

move cursor to mfid1 at the top of the screen<br>
swap 8G (or 4G if there’s a 3rd drive)<br>
/mnt/data2 remaining space <br>
'q' to save and exit

=== distributions ===
Choose the following distribudions

* developer (ok to install ports)
* custom -> lib32
exit

=== media ===
if you are installing via a cd, no need to enter this menu or change anything. Otherwise, choose ftp to install via ftp. You will be prompted to setup networking. You will need to choose a nic (typically bce0 or bce1). Say no to DHCP and IPv6. Hopefully the public nic cable was installed in bce0 so start with that nic and provide the hostname, (public) IP, netmask, gateway and DNS. When configured, it should start pinging. If it doesn't, have the NOC swap cables. Select any FTP server, usually Main or ftp4.

=== commit ===

this usually takes 12mins<br>
during the process you may need to select a new ftp mirror, this is not a problem.<br>
at the conclution of the install you will be prompted to enter the root password (2x) and returned to the configuration menu.

=== add user ===

'''Add''' user 'user'. Defaults for everything is fine, just remember to enter 'wheel' in the member group field.
Do set the password.

=== Setup timezone ===

PT

=== Networking === 
page down to the bottom and enable '[X]' sshd

If you installed via cd, you will need to visit:<br>
interfaces->bce0-><br>
No IPV6<br>
dhcp=no<br>
Set hostname, IP, DNS, gateway<br>
(i.e. setup the nic as indicated above)


Exit the install and if you installed via CD, take it out and let the machine reboot

== Configure OS, kernel, userland, jail ==

=== double check the date/time ===


=== populate /etc/resolv.conf ===

 echo "nameserver 69.55.225.225\
 nameserver 69.55.230.3
 nameserver 69.55.229.3" > /etc/resolv.conf


=== edit /etc/make.conf ===
 echo "WITHOUT_X11=yes \
 KERNCONF=jail3 \
 BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf


=== setup bootloader for console, etc ===

add settings to /boot/loader.conf and /boot.config:

 echo "-Dh" >> /boot.config

 echo 'console="comconsole,vidconsole" \
 boot_multicons="YES" \
 boot_serial="YES" \
 mfi_linux_load="YES" \
 comconsole_speed="115200"' >> /boot/loader.conf


=== enable login via serial console ===
turn off all ttyv's except 0 and 1 in /etc/ttys and turn on ttyd0, change type to vt100:

 vi /etc/ttys

The changed lines should look like:

 ttyv2   "/usr/libexec/getty Pc"         cons25  off secure
 ttyv3   "/usr/libexec/getty Pc"         cons25  off secure
 ttyv4   "/usr/libexec/getty Pc"         cons25  off secure
 ttyv5   "/usr/libexec/getty Pc"         cons25  off secure
 ttyv6   "/usr/libexec/getty Pc"         cons25  off secure
 ttyv7   "/usr/libexec/getty Pc"         cons25  off secure
 # Serial terminals
 # The 'dialup' keyword identifies dialin lines to login, fingerd etc.
 ttyu0   "/usr/libexec/getty std.9600"  vt100   on secure 

Restart init
 kill -1 1

At this point you should have a login on console.

To configure serial console access, login to the console server as root and run:

 # vi /etc/remote

following examples there, rename port to server's hostname, depending on where and which digi box this server is plugged into. Make sure to get speed right too: 115200


=== populate hosts ===

If server is at castle:
 echo "10.1.4.3 backup2" >> /etc/hosts
 echo "10.1.4.8 backup1" >> /etc/hosts

If server is at i2b:
 echo "69.55.230.10 backup2" >> /etc/hosts
 echo "10.1.2.3 backup3" >> /etc/hosts
 echo "69.55.230.11 backup1" >> /etc/hosts


=== create ssh key, upload to backup servers ===
 cd
 ssh-keygen -t dsa -b 1024 
(default location, leave password blank)

If server is at castle:
 cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' 
 cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys'

If server is at i2b:
 cat /root/.ssh/id_dsa.pub | ssh backup3 'cat - >> /root/.ssh/authorized_keys'
 cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' 
 cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys'

confirm that you can ssh to backup2 and backup1 without getting a login prompt

 ssh backup2 hostname
 ssh backup1 hostname
 ssh backup3 hostname


=== create & populate binaries/scripts dirs ===
<pre>mkdir -p /usr/local/jail/bin
mkdir -p /usr/local/jail/rc.d
mkdir -p /usr/local/jail/template/
mkdir /mnt/data1
mkdir /mnt/data2
scp backup2:"/mnt/data4/bin/freebsd8.x/*" /usr/local/jail/bin
cd /usr/local/jail/rc.d/
touch quad1
touch deprecated
chmod +x *
cd /usr/local/jail/bin
ln -s /usr/local/jail/rc.d/quad1 quad1
ln -s /usr/local/jail/bin/jailmake_md jailmake
ln -s /usr/local/jail/bin/js_md js
ln -s /usr/local/jail/bin/canceljail_md canceljail
ln -s /usr/local/jail/bin/jailmakeempty_md jailmakeempty
ln -s /usr/local/jail/bin/postboot_md postboot
ln -s /usr/local/jail/bin/preboot_md preboot
ln -s /usr/local/jail/bin/startjail_md startjail
ln -s /usr/local/jail/bin/stopjail_md stopjail

rehash</pre>


=== edit root's path and login script ===

 vi /root/.cshrc

Change alias entries (add G):

<pre>alias la        ls -aG
alias lf        ls -FAG
alias ll        ls -lAG
alias ls        ls -AG
alias mbm       mb mount
alias mbu       mb umount
alias cjb       cd /usr/local/jail/bin
alias cd1       cd /mnt/data1
alias cd2       cd /mnt/data2
alias cd3       cd /mnt/data3
alias jtop      jtop lj
alias j         jobs</pre>

add to path be careful to leave a space after bin and make sure the wrapping isn't broken:  
 /usr/local/jail/bin 

alter the prompt, set the following:
 set prompt = "`/bin/hostname -s` %/# "

at the bottom of the file add:
<pre>set sshtty=`who am i|awk '{print $2}'`
/usr/sbin/rtprio 3 -`psj | grep $sshtty | awk '{print $2}'`

set shortty=`who am i | awk '{print $2}' | sed -E 's/.*(..)$/\1/'`
foreach x (`psj | grep sh | grep $shortty | awk '{print $2}'`)
/usr/sbin/rtprio 2 -$x
end</pre>

Make the new settings active in current shell:
 source /root/.cshrc


=== install cvsup ===

<pre>cd /usr/ports/net/cvsup-without-gui 
make install clean; rehash; mail -s 'cvs installed' support@johncompanies.com < /dev/null</pre>
stand by for gettext options (use defaults). this process takes approx 22mins- hence the email/page notice above. 


=== get latest sources for this release ===

<pre>cd /usr/src 
echo "*default host=cvsup4.freebsd.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8_3\
*default delete use-rel-suffix\
*default compress\
src-all" > sup</pre>

If you need to run stable (cause release is broken or some other reason) make the sup file look like:

<pre>echo "*default host=cvsup4.freebsd.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8\
*default delete use-rel-suffix\
*default compress\
src-all" > sup</pre>

cvsup sup ; mail -s 'cvs sup done' support@johncompanies.com < /dev/null

time varies, 10-20mins


=== configure new kernel ===

Pull down the kernel config we are using for this distribution. In this case we use an 8.2 kernel config on 8.3, which is valid. The local file should be the same name as host- <tt>jail3</tt> in this example

<pre>cd /usr/src/sys/amd64/conf 
scp backup2:/mnt/data4/build/freebsd/kern_config-8.2-amd64 ./jail3</pre>

edit the kernel config and change ident to be the name of the jail:
 vi jail3
 ident           jail3

Optional, edit <tt>/sys/conf/newvers.sh</tt> to add <tt>–jc2</tt> to the end of the BRANCH string (RELEASE-jc2)
 vi /sys/conf/newvers.sh

notes on kernel configuring: http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html


=== install patches ===

The various patches are in /mnt/data4/build/freebsd/patches on backup2. There are dirs for each version. Not all dirs are populated, but patches for later versions work on older ones unless there is a new patch in the older dir.

there are no patches we use for 8.x, but here would be the commands:

 cd /usr/src
 scp backup2:"/mnt/data4/build/freebsd/patches/8.0/*" .

Apply patches, i.e. the <tt>jls-patch</tt>:
 patch -l < jls-patch


=== build, install kernel and world ===

Rename current generic kernel so it will always be available to boot from. Save room by removing non-needed kernel modules:

<pre>cd /boot
mv kernel kernel.GENERIC
cd kernel.GENERIC
mkdir hold
mv mfi_linux.ko hold/
mv linux.ko hold/
mv linprocfs.ko hold/
mv linsysfs.ko hold/
mv geom_vinum.ko hold/
mv geom_concat.ko hold/
mv zfs.* hold/
mv opensolaris* hold/

rm *.ko
rm *.symbols
mv hold/* .
rmdir hold/

cd /usr/src
make buildkernel installkernel

make buildworld ; mail -s 'buildworld done' dave.boodman@vtext.com < /dev/null</pre>
~38mins

 make installworld 
~34min

 mergemaster -i
You will be prompted to merge, replace or ignore files changed by the src update. In most cases you can '''d'''elete the temp (new) files.

ONLY if this will be a zfs system (not currently used in 8.x):
<pre>cd /sys/modules/zfs
make 
make install
cd /sys/modules/opensolaris
make 
make install</pre>


===  populate devfs ruleset ===
 scp backup2:/mnt/data4/build/freebsd/devfs.rules.8x /etc/devfs.rules


=== populate /etc/rc.conf with IPs and service settings ===

<pre>vi /etc/rc.conf

kern_securelevel_enable="NO"
portmap_enable="NO"
sendmail_enable="NO"
usbd_enable="YES"

nfs_client_enable="YES"
nfs_reserved_port_only="YES"
inetd_enable="YES"
inetd_flags="-wW -a 10.1.4.103"
devfs_system_ruleset="devfsrules_show_all"

ifconfig_bce1="inet 10.1.4.103 netmask 255.255.255.0"
ifconfig_bce0="inet 69.55.229.7 netmask 255.255.255.0"
#ifconfig_bce0_alias0="inet 69.55.2xx.xx netmask 255.255.255.0"

fsck_y_enable="YES"
background_fsck="NO"
#rc_mfi_raid_tty_log="YES"
#zfs_enable="YES"</pre>

Modify IPs, hostname, gateway for this box.

=== make sure sysctls are set and preserved upon boot ===

<pre>echo "kern.consmute=0\
kern.ipc.shm_use_phys=1\
kern.ipc.shmall=131070\
kern.ipc.shmmax=134217728\
net.inet.tcp.syncookies=0\
kern.maxfiles=32768\
kern.fallback_elf_brand=3\
kern.maxprocperuid=4000\
security.jail.sysvipc_allowed=1\
security.jail.allow_raw_sockets=1\
security.jail.socket_unixiproute_only=1\
security.jail.chflags_allowed=0\
dev.amr.0.allow_volume_configure=1\
compat.linux.osrelease=2.6.12\
vm.pmap.shpgperproc=500\
security.bsd.unprivileged_read_msgbuf=0\
kern.maxvnodes=400000" >> /etc/sysctl.conf</pre>

Tuning note: watch vfs.numvnodes while the server is live to get guidance on where to set maxvnodes


=== mount procfs ===
 echo "proc                    /proc           procfs  rw              0       0" >> /etc/fstab
 echo "linprocfs               /usr/compat/linux/proc linprocfs rw     0       0" >> /etc/fstab
 echo "linsysfs                /usr/compat/linux/sys linsysfs rw       0       0" >> /etc/fstab
 mkdir -p /usr/compat/linux/proc
 mkdir -p /usr/compat/linux/sys


=== enable noatime option ===
data1 and data2 should look something like (add ',noatime' after 'rw'):
 /dev/mfid0s1g           /mnt/data1      ufs     rw,noatime      2       2
 /dev/mfid1s1d           /mnt/data2      ufs     rw,noatime      2       2


=== reboot. Confirm new kernel is loaded, devfs in place ===

 uname -a
	
Check devfs rules
 devfs rule showsets
 devfs rule -s 3 show

Should see:
<pre>#  devfs rule showsets
1
2
3
4

#  devfs rule -s 3 show
100 include 1
207 path pts* unhide
217 path fd unhide
218 path fd/* unhide
300 path ttyp* unhide
301 path ttyq* unhide
302 path ttyr* unhide
303 path ttys* unhide
304 path ttyP* unhide
305 path ttyQ* unhide
306 path ttyR* unhide
307 path ttyS* unhide
400 path null unhide
500 path zero unhide
600 path random unhide
610 path urandom unhide
700 path mem unhide
710 path kmem unhide
810 path mdctl unhide
900 path stdin unhide
910 path stdout unhide
920 path stderr unhide</pre>


=== update ports ===

<pre>cd /usr/ports
echo "*default host=cvsup4.FreeBSD.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8_3\
*default delete use-rel-suffix\
*default compress\
ports-all tag=." > sup

cvsup sup; mail -s 'cvs sup ports done' support@johncompanies.com < /dev/null</pre>
~24mins


=== Install raid mgmt tool ===

==== Perc5/i, 6/i ====
Pull over cli from previous system (jail9)
 scp /usr/local/sbin/mega* root@10.1.4.109:/usr/local/sbin/
 scp /usr/local/libexec/MegaCli root@10.1.4.109:/usr/local/libexec/MegaCli
These are linux-based tools. This will require linux base which we would normally install from ports, but since it's failing lately, we just pull in the compat libraries from another system:

 rsync -aSHv --exclude=proc --exclude=sys 10.1.4.109:/usr/compat/linux/ /usr/compat/linux/

Test:
 rehash; megacli ldinfo lall a0

DEPRECATED: Assuming it worked, here's how we ''used to'' install linux_base:
----
 cd /usr/ports/emulators/linux_base-fc4
 make install clean 
(didnt succeed due to libtool requirement)

 cd /usr/ports/distfiles
 fetch http://www.lsi.com/DistributionSystem/AssetDocument/support/downloads/megaraid/miscellaneous/linux/2.00.15_Linux_MegaCLI.zip
 cd /usr/ports/sysutils/linux-megacli
 make install clean
(also failed due to libtool)
----


==== 2850 PERC 4e/Di- no linux ====
<pre>cd /usr/ports/distfiles/
fetch http://backup01.best-hosting.ru/pub/FreeBSD/ports/distfiles/dr_freebsd_1.51.zip
cd /usr/ports/sysutils/megarc
make install clean
megarc -dispCfg -a0</pre>


=== install rsync from ports ===
 cd /usr/ports/net/rsync
 make install clean

choose default options


=== configure inetd to respond to mrtg load queries ===
 echo "load    stream  tcp     nowait  user    /usr/local/jail/bin/load.pl  load.pl" >> /etc/inetd.conf
 echo "load            12384/tcp" >> /etc/services


=== install perl ===
 cd /usr/ports/lang/perl5.12
 make install clean

choose defaults


=== install bb client ===

Compiling from source on AMD64 will not work. So, we use a linux-compiled version and rely on linux compat. Per [[#Perc5.2Fi.2C_6.2Fi|above]], linux compat won't install on 8.x - libtool 2.4 need. So, instead we copy(ed) over linux: 
 rsync -aSHv --exclude=proc --exclude=sys 10.1.4.108:/usr/compat/linux/ /usr/compat/linux/

 adduser

Output/response:

<pre>Username: bb
Full name: bb
Uid (Leave empty for default): 1984
Login group [bb]:
Login group is bb. Invite bb into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: 
Home directory [/home/bb]:
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]: yes
Lock out the account after creation? [no]:
Username   : bb
Password   : <random>
Full Name  : bb
Uid        : 1984
Class      :
Groups     : bb
Home       : /home/bb
Shell      : /bin/sh
Locked     : no
OK? (yes/no): yes</pre>

 cd /usr/home/bb
 scp backup2:/mnt/data4/build/bb/bb-freebsd_linuxcompat.tgz .
 tar xzf bb-freebsd_linuxcompat.tgz

edit /home/bb/bbc1.9e-btf/etc/bb-hosts with something like:

 echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \
 10.1.4.'''103 jail3'''.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts

Edit for machine name and private IP.

if this machine is at i2b:
 echo "69.55.230.2 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \
 69.55.229.7 jail3.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts

<pre>vi /home/bb/bbc1.9e-btf/ext/openfiles 

MACHINE="jail3,johncompanies,com"      # HAS TO BE IN A,B,C FORM</pre>
Edit for machine name.

<pre>cd /usr/home/bb/bbc1.9e-btf/etc
./bbchkcfg.sh 
(y to questions)
./bbchkhosts.sh
(ignore ssh errors)
cd ../..
chown -R bb .
su bb
cd
cd bbc1.9e-btf
./runbb.sh start
more BBOUT 
(look for errors)
exit</pre>

Put in script to start bb @ boot:
 echo 'su - bb -c "cd /home/bb/bbc1.9e-btf; ./runbb.sh start"' > /usr/local/etc/rc.d/bb.sh
 chmod +x /usr/local/etc/rc.d/bb.sh

=== remove reserve space ===
<pre>cd
umount /mnt/data1
umount /mnt/data2
tunefs -m 0 /mnt/data1
tunefs -m 0 /mnt/data2
mount -a</pre>


=== setup rdate ===
 cd /usr/ports/sysutils/rdate
 make install clean

 crontab -e
 0 0 * * * /usr/local/sbin/rdate -s utcnist.colorado.edu

 /usr/local/sbin/rdate -s utcnist.colorado.edu

We used to use ntpd, however it listens on jail IPs which is a security risk. So we stopped. Here's the old instructions:
install new ntp from ports
 /usr/ports/net/ntp
 make install clean

<pre>echo "server 10.1.4.5" > /etc/ntp.conf
/usr/sbin/ntpd -p /var/run/ntpd.pid
sleep 2; ntpq -p</pre>(confirm it’s able to reach our time server)


=== make a data partition ===

IF you didn't format the data partition during sysinstall:

Create a g partition on 2nd mirror – bsdlabel no longer works (below shows d partition made with sysinstall):

<pre>jail8 /usr/home/bb# gpart show
=>       63  285474735  mfid0  MBR  (136G)
         63  285458922      1  freebsd  [active]  (136G)
  285458985      15813         - free -  (7.7M)

=>        0  285458922  mfid0s1  BSD  (136G)
          0     524288        1  freebsd-ufs  (256M)
     524288   12582912        2  freebsd-swap  (6.0G)
   13107200     524288        4  freebsd-ufs  (256M)
   13631488     524288        5  freebsd-ufs  (256M)
   14155776    8388608        6  freebsd-ufs  (4.0G)
   22544384  262914538        7  freebsd-ufs  (125G)

=>       63  584843175  mfid1  MBR  (279G)
         63  584830197      1  freebsd  [active]  (279G)
  584830260      12978         - free -  (6.3M)

=>        0  584830197  mfid1s1  BSD  (279G)
          0   16777216        2  freebsd-swap  (8.0G)
   16777216  568052981        4  freebsd-ufs  (271G)

jail8 /usr/home/bb# gpart show mfid1s1
=>        0  584830197  mfid1s1  BSD  (279G)
          0   16777216        2  freebsd-swap  (8.0G)
   16777216  568052981        4  freebsd-ufs  (271G)

# gpart list mfid1s1

Geom name: mfid1s1
fwheads: 255
fwsectors: 63
last: 584830196
first: 0
entries: 8
scheme: BSD
Providers:
1. Name: mfid1s1b
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Mode: r1w1e0
   rawtype: 1
   length: 8589934592
   offset: 0
   type: freebsd-swap
   index: 2
   end: 16777215
   start: 0
2. Name: mfid1s1d
   Mediasize: 290843126272 (271G)
   Sectorsize: 512
   Mode: r0w0e0
   rawtype: 7
   length: 290843126272
   offset: 8589934592
   type: freebsd-ufs
   index: 4
   end: 584830196
   start: 16777216
Consumers:
1. Name: mfid1s1
   Mediasize: 299433060864 (279G)
   Sectorsize: 512
   Mode: r1w1e1

# gpart delete -i 4 mfid1s1
mfid1s1d deleted
jail8 /usr/home/bb# gpart list mfid1s1
Geom name: mfid1s1
fwheads: 255
fwsectors: 63
last: 584830196
first: 0
entries: 8
scheme: BSD
Providers:
1. Name: mfid1s1b
   Mediasize: 8589934592 (8.0G)
   Sectorsize: 512
   Mode: r1w1e0
   rawtype: 1
   length: 8589934592
   offset: 0
   type: freebsd-swap
   index: 2
   end: 16777215
   start: 0
Consumers:
1. Name: mfid1s1
   Mediasize: 299433060864 (279G)
   Sectorsize: 512
   Mode: r1w1e1

# gpart show mfid1s1
=>        0  584830197  mfid1s1  BSD  (279G)
          0   16777216        2  freebsd-swap  (8.0G)
   16777216  568052981           - free -  (271G)

# gpart add  -t freebsd-ufs -i 7 mfid1s1
mfid1s1g added

# gpart show mfid1s1
=>        0  584830197  mfid1s1  BSD  (279G)
          0   16777216        2  freebsd-swap  (8.0G)
   16777216  568052981        7  freebsd-ufs  (271G)</pre>

Here's how we USED to do it with bsdlabel:

 bsdlabel -e /dev/mfid0s1

given:
<pre># /dev/aacd0s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:   262144        0    4.2BSD     2048 16384 16392
  b:  4194304   262144      swap
  c: 143363997        0    unused        0     0         # "raw" part, don't edit
  d:   524288  4456448    4.2BSD     2048 16384 32776
  e:   524288  4980736    4.2BSD     2048 16384 32776
  f:  6291456  5505024    4.2BSD     2048 16384 28552</pre>

new offset = 6291456 + 5505024 = 11796480<br>
new size is size for 'c' partition minus the new start from above<br>
143363997 - 11796480 = 131567517

So:<br>
 g: 131567517 11796480 unused 0 0


=== create the jail template ===

cd /usr/ports/sysutils/jailutils
make install clean

Create an md device to hold the jail:
<pre>touch /mnt/data1/jail-template20g
mdconfig -a -t vnode -s 20g -f /mnt/data1/jail-template20g -u 0
newfs -O 1 /dev/md0
mkdir /mnt/data1/jail-DIR
mount /dev/md0 /mnt/data1/jail-DIR</pre>

Build world into the jail:
<pre>cd /usr/src
make world DESTDIR=/mnt/data1/jail-DIR; taskdone</pre>
~1hr

Make /etc into the jail, mount dev, copy in jkill:
<pre>cd etc
make distribution DESTDIR=/mnt/data1/jail-DIR
mount -t devfs devfs /mnt/data1/jail-DIR/dev
devfs -m /mnt/data1/jail-DIR/dev rule -s 3 applyset 
cd /mnt/data1/jail-DIR
ln -sf dev/null kernel
cp /usr/local/sbin/jkill /mnt/data1/jail-DIR/sbin</pre>

Enter into jail to do configuration:
 jail /mnt/data1/jail-DIR testhostname 192.168.11.100 /bin/sh
 csh

Create fstab:
<pre>touch /etc/fstab
echo 'network_interfaces=""\
hostname="newsystem"\
kern_securelevel_enable="NO"\
sendmail_enable="YES"\
sshd_enable="YES"' > /etc/rc.conf

echo "nameserver 69.55.225.225\
nameserver 69.55.230.3" >> /etc/resolv.conf</pre>

Edit crontab:
 vi /etc/crontab
remove the adjkerntz lines
comment out periodic’s and put this line above them:
 # IF YOU UNCOMMENT THESE, PLEASE ADJUST THEIR RUN TIME

 rm -rf /etc/periodic/daily/400.status-disks

check and remove any crap in /tmp

 vi /etc/periodic/security/100.chksetuid
replace: <tt>MP=`mount -t ufs | grep -v " nosuid" | awk '{ print $3 }' | sort`</tt><br>
with: <tt>MP='/'</tt> (use single quotes)

 mkdir -p /usr/compat/linux/dev

Add account for user. Output/response:
<pre>adduser

Username: user
Full name: user
Uid (Leave empty for default):
Login group [user]:
Login group is user. Invite user into other groups? []: wheel
Login class [default]:
Shell (sh csh tcsh nologin) [sh]:
Home directory [/home/user]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]: y
Lock out the account after creation? [no]:
Username   : user
Password   : <random>
Full Name  : user
Uid        : 1001
Class      :
Groups     : user
Home       : /home/user
Home Mode  :
Shell      : /bin/sh
Locked     : no
OK? (yes/no): y
adduser: INFO: Successfully added (user) to the user database.
adduser: INFO: Password for (user) is: 901gmYjO
Add another user? (yes/no): n
Goodbye!</pre>

set TERM:
 vi /usr/home/user/.profile 
 TERM=vt100;     export TERM

Set time zone to PT:
 tzsetup

Reload aliases:
 newaliases 

Replace reboot/halt:
 rm /sbin/halt /sbin/reboot
 ln /sbin/jkill /sbin/halt
 ln /sbin/jkill /sbin/reboot

Redirect console output, comment out console and move to /var/log/messages:
 vi /etc/syslog.conf
 #*.err;kern.warning;auth.notice;mail.crit               /dev/console
 *.err;kern.warning;auth.notice;mail.crit                /var/log/messages 

 exit
 exit

Lib32 compat library:
 cd libexec
 chflags noschg ld-elf32.so.1
 chflags noschg ld-elf.so.1
 mv ld-elf32.so.1 ld-elf32.so.1-orig
 ln ld-elf.so.1 ld-elf32.so.1
 chflags schg ld-elf.so.1
 chflags schg ld-elf32.so.1

Replace traceroute:
 mv /mnt/data1/jail-DIR/usr/sbin/traceroute /mnt/data1/jail-DIR/usr/sbin/_traceroute
 echo '#\!/bin/sh\
 /usr/sbin/_traceroute -i bce0 $1' >> /mnt/data1/jail-DIR/usr/sbin/traceroute 
 chmod +x /mnt/data1/jail-DIR/usr/sbin/traceroute 

Modify 'bce0' to reflect whichever nic is public on this hardware.

Clean out ports before copying into the jail (~30mins):
 cd /usr/ports
 make -DNOCLEANDEPENDS clean 
 rm -fr /usr/ports/distfiles/*
 cp -r /usr/ports /mnt/data1/jail-DIR/usr 

 rm /mnt/data1/jail-DIR/root/.history

Umount the jail and dump it:
<pre>cd /
umount /mnt/data1/jail-DIR/dev
dump -0a -f /usr/local/jail/template/template /dev/md0
umount /dev/md0
rmdir /mnt/data1/jail-DIR
mdconfig -d -u 0</pre>


=== setup backups ===
<pre>echo '#\!/bin/sh\
backupdir=/data/jail3\
server=backup1\
\
## ENTRY /etc\
## ENTRY /usr/local/etc\
## ENTRY /usr/local/jail\
## ENTRY /root/logs' > /usr/local/jail/bin/backup.config</pre>

Edit to reflect backup server and jail hostname

On backup server, setup backup dirs:
 ssh backup1 mkdir -p /data/jail3/0

On backup server, setup backup dirs:
 backup1# vi /usr/local/sbin/snapshot_rotate

Edit /usr/local/jail/bin/backup.xxx to use the right drives and copy to /usr/local/jail/bin/backup 
 vi /usr/local/jail/bin/backup.md
adjust df so it includes all relevant drives, currently <tt>df > /etc/df.bak</tt> is fine. Also, make sure the binary/script source is pulling from the right dir on backup2:
 /usr/local/bin/rsync -a backup2:/mnt/data4/bin/freebsd8.x/ /usr/local/jail/bin/

 cp /usr/local/jail/bin/backup.md /usr/local/jail/bin/backup


=== create /root/logs ===
 mkdir /root/logs


=== edit sshd_config ===
 vi /etc/ssh/sshd_config
 ListenAddress 69.55.229.7
 ListenAddress 10.1.4.103

'''''Adjust to pub/private IP for jail.'''
''
Restart sshd:
 kill -1 `cat /var/run/sshd.pid`

=== add crontab entries ===
<pre>crontab -e
5 0 * * * /usr/local/jail/bin/backup
1 0 1 * * /usr/local/jail/bin/ipfwreset
0 18 * * * /usr/local/jail/bin/ipfwbackup
4,9,14,19,24,29,34,39,44,49,55,59 * * * * /usr/local/jail/bin/trafstats
0 0,6,12,18 * * * /usr/local/jail/bin/sync_jail_names</pre>

On 2950:
 */5 * * * * /usr/local/jail/bin/perc5iraidchk

On 3ware-based servers:
 0 0 * * * /usr/local/jail/bin/3wraidchk


=== Reboot notify script ===
 ln -s /usr/local/jail/bin/notify.sh /usr/local/etc/rc.d/notify.sh


=== patch jail against starting jails with rtprio ===

<pre>mv /usr/sbin/jail /usr/sbin/jail_
echo '#\!/bin/sh\
/usr/sbin/rtprio -t /usr/sbin/jail_ $*' > /usr/sbin/jail
chmod +x /usr/sbin/jail</pre>


=== make sure mail works ===
If there are map errors:
 cd /etc/mail; make maps


=== recover space on /usr (optional) ===

 rm -fr /usr/obj


=== wrapper for jps ===

 mv /usr/local/sbin/jps /usr/local/sbin/jps_


=== wrapper for jls ===

 mv /usr/sbin/jls /usr/sbin/jls_


=== wrapper for jexec ===

 mv /usr/sbin/jexec /usr/sbin/jexec_


=== install jtop ===

 cd /usr/ports/sysutils/jtop
 make install clean


=== block jails from reaching private net ===
 echo 'ipfw add 1 deny ip from 69.55.224.0/20 to 10.1.4.0/24' > /usr/local/etc/rc.d/ipfw.sh
 chmod 0700 /usr/local/etc/rc.d/ipfw.sh

== add to management infrastructure ==

=== mail ===

==== add to management db ====

tables: jc.ref_machines and jc.ref_templates

on jail run:
 uname -r

Which shows something like:
 8.3-RELEASE-p2

Insert into db:

<pre>insert into ref_machines values (null,'jail3','mx3.johncompanies.com',0,'f8');
select machine_id from ref_machines where host='jail3';
+------------+
| machine_id |
+------------+
|         35 |
+------------+
insert into ref_templates values ('','8.3-RELEASE-jc2',35,'FreeBSD 8.3',1);</pre>

==== add to bb server ====
 vi /usr/home/bb/bbsrc/bb1.9i-btf/etc/bb-hosts
 10.1.4.109 jail9.johncompanies.com # ssh

In the case of an i2b server, use real ip:
 69.55.229.7 jail3.johncompanies.com # ssh

<pre>su bb
cd
bbsrc/bb/runbb.sh restart ; exit</pre>

==== Update backupgraph ====
 vi /usr/local/www/mgmt/cgi/backupgraph.pl
(add hostname)

==== Update load mrtg ====
 vi /usr/local/www/mgmt/mrtg/mrtg1.cfg 
(add new entry to file following existing format)

=== ns1c ===
fwd and reverse lookups:
 vr johncompanies.com

 vi internal.johncompanies.com
 rndc reload johncompanies.com IN private

 ptr 69.55.227.x

=== wiki ===
add to server/cabinet map

=== firewall ===
add an outside blocking rule to the firewall, so this machine can only be reached from inside the firewall. Follow example already in firewall jail17 is:
 
 00117 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.228.200
 00117 deny ip from any to 69.55.228.200

jail19 would be 00119...
 ipfw add 00109 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 } to 69.55.232.3
 ipfw add 00109 deny ip from any to 69.55.232.3

=== select some customers for castle probe map ===