Jail Server Install

8.x

Assumptions

Setup instructions below assume this is DELL 2950 with an LSI-based SAS RAID card.

Server is at castle, connected to pub, private, serial and DRAC

Assuming load done via IPKVM with ISO mounted via USB

Assumes at least 4 drives, 2 mirrors

Configure server BIOS

setup console redirect, speed 115200

set LCD string to name of server "jail8"

set date to GMT

go into RAID bios and setup mirrors

configure DRAC: TODO

Install OS (sysinstall)

boot to bootonly disk for AMD version of FreeBSD, i.e. FreeBSD-8.3-RELEASE-amd64-bootonly.iso

when the install menu appears, choose custom install

partition menu

move cursor to mfid0, hit space (takes you to partition map screen). If there is only 1 mirror, there will be no option to select a drive.

type 'a' to use entire disk
type 'q' to quit and save
choose 'freebsd standard mbr'

space to unselect mfid0
cursor down to mfid1
hit space

type 'a' to use entire disk
type 'q' to quit and save
choose 'none' for boot mgr (leave untouched)

cursor over mfid0
space
(takes you into part. Screen again) 'q' to exit
none for boot mgr

Make sure both drives (mfid0 and mfid1) are checked and tab to ok

Label menu

Make sure mfid0 is highlighted at the top of the screen, setup the following partitions

/ 512M
swap 6G
/var 256M
/tmp 256M
/usr 5G
/mnt/data1 remaining space

All partitions except / should be setup for soft updates. If not, type 's' to enable for soft updates on all except for / (should look like UFS2+S Y under the Newfs column)

move cursor to mfid1 at the top of the screen
swap 8G (or 4G if there’s a 3rd drive)
/mnt/data2 remaining space
'q' to save and exit

distributions

Choose the following distribudions

• developer (ok to install ports)
• custom -> lib32

exit

media

if you are installing via a cd, no need to enter this menu or change anything. Otherwise, choose ftp to install via ftp. You will be prompted to setup networking. You will need to choose a nic (typically bce0 or bce1). Say no to DHCP and IPv6. Hopefully the public nic cable was installed in bce0 so start with that nic and provide the hostname, (public) IP, netmask, gateway and DNS. When configured, it should start pinging. If it doesn't, have the NOC swap cables. Select any FTP server, usually Main or ftp4.

commit

this usually takes 12mins
during the process you may need to select a new ftp mirror, this is not a problem.
at the conclution of the install you will be prompted to enter the root password (2x) and returned to the configuration menu.

add user

Add user 'user'. Defaults for everything is fine, just remember to enter 'wheel' in the member group field. Do set the password.

Setup timezone

PT

Networking

page down to the bottom and enable '[X]' sshd

If you installed via cd, you will need to visit:
interfaces->bce0->
No IPV6
dhcp=no
Set hostname & domain
(i.e. setup the nic as indicated above)

Exit the install and if you installed via CD, take it out and let the machine reboot

Configure OS, kernel, userland

edit /etc/make.conf

echo "WITHOUT_X11=yes \
KERNCONF=jail3 \
BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf

setup bootloader for console, etc

add settings to /boot/loader.conf and /boot.config:

echo "-Dh" >> /boot.config

echo 'console="comconsole,vidconsole" \
boot_multicons="YES" \
boot_serial="YES" \
mfi_linux_load="YES" \
comconsole_speed="115200"' >> /boot/loader.conf

enable login via serial console

turn off all ttyv's except 0 and 1 in /etc/ttys and turn on ttyd0, change type to vt100:

vi /etc/ttys

The changed lines should look like:

ttyv2   "/usr/libexec/getty Pc"         cons25  off secure
ttyv3   "/usr/libexec/getty Pc"         cons25  off secure
ttyv4   "/usr/libexec/getty Pc"         cons25  off secure
ttyv5   "/usr/libexec/getty Pc"         cons25  off secure
ttyv6   "/usr/libexec/getty Pc"         cons25  off secure
ttyv7   "/usr/libexec/getty Pc"         cons25  off secure
# Serial terminals
# The 'dialup' keyword identifies dialin lines to login, fingerd etc.
ttyu0   "/usr/libexec/getty std.9600"  vt100   on secure 

Restart init

kill -1 1

At this point you should have a login on console.

To configure this server on the console server, as root:

# vi /etc/remote

following examples there, rename port to jail8 depending on where and which digi box this server is plugged into. Make sure to get speed right: 115200

populate hosts

echo "10.1.4.3 backup2" >> /etc/hosts
echo "10.1.4.8 backup1" >> /etc/hosts
echo "10.1.2.3 backup3" >> /etc/hosts

create ssh key, upload to backup servers

cd
ssh-keygen -t dsa -b 1024 

(default location, leave password blank)

cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' 
cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys'
cat /root/.ssh/id_dsa.pub | ssh backup3 'cat - >> /root/.ssh/authorized_keys'

confirm that you can ssh to backup2 and backup1 without getting a login prompt

ssh backup2 hostname
ssh backup1 hostname
ssh backup3 hostname

create & populate binaries/scripts dirs

mkdir -p /usr/local/jail/bin
mkdir -p /usr/local/jail/rc.d
mkdir -p /usr/local/jail/template/
mkdir /mnt/data1
mkdir /mnt/data2
scp backup2:"/mnt/data4/bin/freebsd8.x/*" /usr/local/jail/bin
cd /usr/local/jail/rc.d/
touch quad1
touch deprecated
chmod +x *
cd /usr/local/jail/bin
ln -s /usr/local/jail/rc.d/quad1 quad1
ln -s /usr/local/jail/bin/jailmake_md jailmake
ln -s /usr/local/jail/bin/js_md js
ln -s /usr/local/jail/bin/canceljail_md canceljail
ln -s /usr/local/jail/bin/jailmakeempty_md jailmakeempty
ln -s /usr/local/jail/bin/postboot_md postboot
ln -s /usr/local/jail/bin/preboot_md preboot
ln -s /usr/local/jail/bin/startjail_md startjail
ln -s /usr/local/jail/bin/stopjail_md stopjail

rehash

edit root's path and login script

vi /root/.cshrc

Change alias entries (add G):

alias la        ls -aG
alias lf        ls -FAG
alias ll        ls -lAG
alias ls        ls -AG
alias mbm       mb mount
alias mbu       mb umount
alias cjb       cd /usr/local/jail/bin
alias cd1       cd /mnt/data1
alias cd2       cd /mnt/data2
alias cd3       cd /mnt/data3
alias jtop      jtop lj
alias j         jobs

add to path be careful to leave a space after bin and make sure the wrapping isn't broken:

/usr/local/jail/bin 

alter the prompt, set the following:

set prompt = "`/bin/hostname -s` %/# "

at the bottom of the file add:

set sshtty=`who am i|awk '{print $2}'`
/usr/sbin/rtprio 3 -`psj | grep $sshtty | awk '{print $2}'`

set shortty=`who am i | awk '{print $2}' | sed -E 's/.*(..)$/\1/'`
foreach x (`psj | grep sh | grep $shortty | awk '{print $2}'`)
/usr/sbin/rtprio 2 -$x
end

Make the new settings active in current shell:

source /root/.cshrc

install cvsup

cd /usr/ports/net/cvsup-without-gui 
make install clean; rehash; mail -s 'cvs installed' support@johncompanies.com < /dev/null

stand by for gettext options (use defaults). this process takes approx 22mins- hence the email/page notice above.

get latest sources for this release

cd /usr/src 
echo "*default host=cvsup4.freebsd.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8_3\
*default delete use-rel-suffix\
*default compress\
src-all" > sup

If you need to run stable (cause release is broken or some other reason) make the sup file look like:

echo "*default host=cvsup4.freebsd.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8\
*default delete use-rel-suffix\
*default compress\
src-all" > sup

cvsup sup ; mail -s 'cvs sup done' support@johncompanies.com < /dev/null

time varies, 10-20mins

configure new kernel

Pull down the kernel config we are using for this distribution. In this case we use an 8.2 kernel config on 8.3, which is valid. The local file should be the same name as host- jail3 in this example

cd /usr/src/sys/amd64/conf 
scp backup2:/mnt/data4/build/freebsd/kern_config-8.2-amd64 ./jail3

edit the kernel config and change ident to be the name of the jail:

vi jail3
ident           jail3

Optional, edit /sys/conf/newvers.sh to add –jc2 to the end of the BRANCH string (RELEASE-jc2)

vi /sys/conf/newvers.sh

notes on kernel configuring: http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html

install patches

The various patches are in /mnt/data4/build/freebsd/patches on backup2. There are dirs for each version. Not all dirs are populated, but patches for later versions work on older ones unless there is a new patch in the older dir.

there are no patches we use for 8.x, but here would be the commands:

cd /usr/src
scp backup2:"/mnt/data4/build/freebsd/patches/8.0/*" .

Apply patches, i.e. the jls-patch:

patch -l < jls-patch

build, install kernel and world

Rename current generic kernel so it will always be available to boot from. Save room by removing non-needed kernel modules:

cd /boot
mv kernel kernel.GENERIC
cd kernel.GENERIC
mkdir hold
mv mfi_linux.ko hold/
mv linux.ko hold/
mv linprocfs.ko hold/
mv linsysfs.ko hold/
mv geom_vinum.ko hold/
mv geom_concat.ko hold/
mv zfs.* hold/
mv opensolaris* hold/

rm *.ko
rm *.symbols
mv hold/* .
rmdir hold/

cd /usr/src
make buildkernel installkernel

make buildworld ; mail -s 'buildworld done' dave.boodman@vtext.com < /dev/null

~38mins

make installworld 

~34min

mergemaster -i

You will be prompted to merge, replace or ignore files changed by the src update. In most cases you can delete the temp (new) files.

ONLY if this will be a zfs system (not currently used in 8.x):

cd /sys/modules/zfs
make 
make install
cd /sys/modules/opensolaris
make 
make install

populate devfs ruleset

scp backup2:/mnt/data4/build/freebsd/devfs.rules.8x /etc/devfs.rules

populate /etc/rc.conf with IPs and service settings

vi /etc/rc.conf

kern_securelevel_enable="NO"
portmap_enable="NO"
sendmail_enable="NO"
usbd_enable="YES"

nfs_client_enable="YES"
nfs_reserved_port_only="YES"
inetd_enable="YES"
inetd_flags="-wW -a 10.1.2.103"
devfs_system_ruleset="devfsrules_show_all"

ifconfig_bce1="inet 10.1.2.103 netmask 255.255.255.0"
ifconfig_bce0="inet 69.55.229.7 netmask 255.255.255.0"
#ifconfig_bce0_alias0="inet 69.55.2xx.xx netmask 255.255.255.0"

fsck_y_enable="YES"
background_fsck="NO"
#rc_mfi_raid_tty_log="YES"
#zfs_enable="YES"

Modify IPs, hostname, gateway for this box.

make sure sysctls are set and preserved upon boot

echo "kern.consmute=0\
kern.ipc.shm_use_phys=1\
kern.ipc.shmall=131070\
kern.ipc.shmmax=134217728\
net.inet.tcp.syncookies=0\
kern.maxfiles=32768\
kern.fallback_elf_brand=3\
kern.maxprocperuid=4000\
security.jail.sysvipc_allowed=1\
security.jail.allow_raw_sockets=1\
security.jail.socket_unixiproute_only=1\
security.jail.chflags_allowed=0\
dev.amr.0.allow_volume_configure=1\
compat.linux.osrelease=2.6.12\
vm.pmap.shpgperproc=500\
security.bsd.unprivileged_read_msgbuf=0\
kern.maxvnodes=400000" >> /etc/sysctl.conf

Tuning note: watch vfs.numvnodes while the server is live to get guidance on where to set maxvnodes

mount procfs

echo "proc                    /proc           procfs  rw              0       0" >> /etc/fstab
echo "linprocfs               /usr/compat/linux/proc linprocfs rw     0       0" >> /etc/fstab
echo "linsysfs                /usr/compat/linux/sys linsysfs rw       0       0" >> /etc/fstab
mkdir -p /usr/compat/linux/proc
mkdir -p /usr/compat/linux/sys

enable noatime option

data1 and data2 should look something like (add ',noatime' after 'rw'):

/dev/mfid0s1g           /mnt/data1      ufs     rw,noatime      2       2
/dev/mfid1s1d           /mnt/data2      ufs     rw,noatime      2       2

reboot. Confirm new kernel is loaded, devfs in place

uname -a

Check devfs rules

devfs rule showsets
devfs rule -s 3 show

Should see:

#  devfs rule showsets
1
2
3
4

#  devfs rule -s 3 show
100 include 1
207 path pts* unhide
217 path fd unhide
218 path fd/* unhide
300 path ttyp* unhide
301 path ttyq* unhide
302 path ttyr* unhide
303 path ttys* unhide
304 path ttyP* unhide
305 path ttyQ* unhide
306 path ttyR* unhide
307 path ttyS* unhide
400 path null unhide
500 path zero unhide
600 path random unhide
610 path urandom unhide
700 path mem unhide
710 path kmem unhide
810 path mdctl unhide
900 path stdin unhide
910 path stdout unhide
920 path stderr unhide

update ports

cd /usr/ports
echo "*default host=cvsup4.FreeBSD.org\
*default base=/usr\
*default prefix=/usr\
*default release=cvs tag=RELENG_8_3\
*default delete use-rel-suffix\
*default compress\
ports-all tag=." > sup

cvsup sup; mail -s 'cvs sup ports done' support@johncompanies.com < /dev/null

~24mins

22. Install raid mgmt tool

(for 2850 PERC 4e/Di- no linux) cd /usr/ports/distfiles/ fetch http://backup01.best-hosting.ru/pub/FreeBSD/ports/distfiles/dr_freebsd_1.51.zip cd /usr/ports/sysutils/megarc make install clean megarc -dispCfg -a0

need to install perl since linux base won't grab it: cd /usr/ports/lang/perl5.8 make install clean

---

(for Perc5/i, 6/i) install linux_base: cd /usr/ports/emulators/linux_base-fc4 make install clean (2450: 7min, supermicro: 3mins, 2950: 14mins) Note: didnt succeed due to libtool requirement

cd /usr/ports/distfiles fetch http://www.lsi.com/DistributionSystem/AssetDocument/support/downloads/megaraid/miscellaneous/linux/2.00.15_Linux_MegaCLI.zip cd /usr/ports/sysutils/linux-megacli make install clean also failed due to libtool, so did scp /usr/local/sbin/mega* root@10.1.4.110:/usr/local/sbin/ scp /usr/local/libexec/MegaCli root@10.1.4.110:/usr/local/libexec/MegaCli

Test: rehash; megacli ldinfo lall a0

23. install rsync from ports cd /usr/ports/net/rsync make install clean

choose default options

24. configure inetd to respond to mrtg load queries echo "load stream tcp nowait user /usr/local/jail/bin/load.pl load.pl" >> /etc/inetd.conf

echo "load 12384/tcp" >> /etc/services

25. install bb client (need linux compat for this, won't install on 8.2 - libtool 2.4 need. So, instead copied over linux: rsync -aSHv --exclude=proc --exclude=sys 10.1.4.108:/usr/compat/linux/ /usr/compat/linux/) NEED TO INSTALL PERL SEPARATELY!

adduser Username: bb Full name: bb Uid (Leave empty for default): 1984 Login group [bb]: Login group is bb. Invite bb into other groups? []: Login class [default]: Shell (sh csh tcsh nologin) [sh]: Home directory [/home/bb]: Use password-based authentication? [yes]: Use an empty password? (yes/no) [no]: Use a random password? (yes/no) [no]: yes Lock out the account after creation? [no]: Username  : bb Password  : <random> Full Name  : bb Uid  : 1984 Class  : Groups  : bb Home  : /home/bb Shell  : /bin/sh Locked  : no OK? (yes/no): yes

cd /usr/home/bb scp backup2:/mnt/data4/build/bb/bb-freebsd_linuxcompat.tgz . tar xzf bb-freebsd_linuxcompat.tgz

edit /home/bb/bbc1.9e-btf/etc/bb-hosts with something like: echo "69.55.230.2 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \ 69.55.229.7 jail3.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts or echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \ 10.1.4.103 jail3.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts

vi /home/bb/bbc1.9e-btf/ext/openfiles MACHINE="jail3,johncompanies,com" # HAS TO BE IN A,B,C FORM

cd /usr/home/bb/bbc1.9e-btf/etc ./bbchkcfg.sh (y to questions) ./bbchkhosts.sh (ignore ssh errors) cd ../.. chown -R bb . su bb cd cd bbc1.9e-btf ./runbb.sh start more BBOUT (look for errors) exit

echo 'su - bb -c "cd /home/bb/bbc1.9e-btf; ./runbb.sh start"' > /usr/local/etc/rc.d/bb.sh chmod +x /usr/local/etc/rc.d/bb.sh

NOTE: to get bb working on amd, had to copy over bin dir from linux dist

26. configure load mrtg, on mail vi /usr/local/www/mgmt/mrtg/mrtg1.cfg (add new entry to file following existing format)

27. configure bb on mail: vi /usr/home/bb/bbsrc/bb1.9i-btf/etc/bb-hosts 10.1.4.109 jail9.johncompanies.com # ssh or 69.55.229.7 jail3.johncompanies.com # ssh

su bb cd bbsrc/bb/runbb.sh restart ; exit

28. remove reserve space, enable softupdates (probably already set, so not necessary) cd umount /mnt/data1 umount /mnt/data2 tunefs -m 0 /mnt/data1 tunefs -m 0 /mnt/data2 mount -a

29. DEPRECATED - ntpd listens on jail IPs- security risk echo "server 10.1.4.5" > /etc/ntp.conf

/usr/sbin/ntpd -p /var/run/ntpd.pid sleep 2; ntpq -p (confirm it’s able to reach our time server)

But there's a bug so install new ntp from ports /usr/ports/net/ntp

30. fwd and reverse lookups on ns1c vr johncompanies.com vi internal.johncompanies.com rndc reload johncompanies.com IN private (edit the PTR too)

31. if needed, make a g partition

bsdlabel -e /dev/mfid0s1

given:

1. /dev/aacd0s1:

8 partitions:

1. size offset fstype [fsize bsize bps/cpg]

 a:   262144        0    4.2BSD     2048 16384 16392
 b:  4194304   262144      swap
 c: 143363997        0    unused        0     0         # "raw" part, don't edit
 d:   524288  4456448    4.2BSD     2048 16384 32776
 e:   524288  4980736    4.2BSD     2048 16384 32776
 f:  6291456  5505024    4.2BSD     2048 16384 28552

new offset = 6291456 + 5505024 = 11796480 new size is size for 'c' partition minus the new start from above 143363997 - 11796480 = 131567517 So: g: 131567517 11796480 unused 0 0

Create a g partition on 2nd mirror – bsdlabel no longer works (below shows d partition made with sysinstall):

jail8 /usr/home/bb# gpart show => 63 285474735 mfid0 MBR (136G)

        63  285458922      1  freebsd  [active]  (136G)
 285458985      15813         - free -  (7.7M)

=> 0 285458922 mfid0s1 BSD (136G)

         0     524288        1  freebsd-ufs  (256M)
    524288   12582912        2  freebsd-swap  (6.0G)
  13107200     524288        4  freebsd-ufs  (256M)
  13631488     524288        5  freebsd-ufs  (256M)
  14155776    8388608        6  freebsd-ufs  (4.0G)
  22544384  262914538        7  freebsd-ufs  (125G)

=> 63 584843175 mfid1 MBR (279G)

        63  584830197      1  freebsd  [active]  (279G)
 584830260      12978         - free -  (6.3M)

=> 0 584830197 mfid1s1 BSD (279G)

         0   16777216        2  freebsd-swap  (8.0G)
  16777216  568052981        4  freebsd-ufs  (271G)

jail8 /usr/home/bb# gpart show mfid1s1 => 0 584830197 mfid1s1 BSD (279G)

         0   16777216        2  freebsd-swap  (8.0G)
  16777216  568052981        4  freebsd-ufs  (271G)

1. gpart list mfid1s1

Geom name: mfid1s1 fwheads: 255 fwsectors: 63 last: 584830196 first: 0 entries: 8 scheme: BSD Providers: 1. Name: mfid1s1b

  Mediasize: 8589934592 (8.0G)
  Sectorsize: 512
  Mode: r1w1e0
  rawtype: 1
  length: 8589934592
  offset: 0
  type: freebsd-swap
  index: 2
  end: 16777215
  start: 0

2. Name: mfid1s1d

  Mediasize: 290843126272 (271G)
  Sectorsize: 512
  Mode: r0w0e0
  rawtype: 7
  length: 290843126272
  offset: 8589934592
  type: freebsd-ufs
  index: 4
  end: 584830196
  start: 16777216

Consumers: 1. Name: mfid1s1

  Mediasize: 299433060864 (279G)
  Sectorsize: 512
  Mode: r1w1e1

1. gpart delete -i 4 mfid1s1

mfid1s1d deleted jail8 /usr/home/bb# gpart list mfid1s1 Geom name: mfid1s1 fwheads: 255 fwsectors: 63 last: 584830196 first: 0 entries: 8 scheme: BSD Providers: 1. Name: mfid1s1b

  Mediasize: 8589934592 (8.0G)
  Sectorsize: 512
  Mode: r1w1e0
  rawtype: 1
  length: 8589934592
  offset: 0
  type: freebsd-swap
  index: 2
  end: 16777215
  start: 0

Consumers: 1. Name: mfid1s1

  Mediasize: 299433060864 (279G)
  Sectorsize: 512
  Mode: r1w1e1

1. gpart show mfid1s1

=> 0 584830197 mfid1s1 BSD (279G)

         0   16777216        2  freebsd-swap  (8.0G)
  16777216  568052981           - free -  (271G)

1. gpart add -t freebsd-ufs -i 7 mfid1s1

mfid1s1g added

1. gpart show mfid1s1

=> 0 584830197 mfid1s1 BSD (279G)

         0   16777216        2  freebsd-swap  (8.0G)
  16777216  568052981        7  freebsd-ufs  (271G)

32. create the jail template

touch /mnt/data1/jail-template20g mdconfig -a -t vnode -s 20g -f /mnt/data1/jail-template20g -u 0 newfs -O 1 /dev/md0 mkdir /mnt/data1/jail-DIR mount /dev/md0 /mnt/data1/jail-DIR

cd /usr/ports/sysutils/jailutils make install clean

cd /usr/src make world DESTDIR=/mnt/data1/jail-DIR; pagedave (2450: 2:28mins, supermicro: 55mins, 2950: 1h ) cd etc make distribution DESTDIR=/mnt/data1/jail-DIR mount -t devfs devfs /mnt/data1/jail-DIR/dev devfs -m /mnt/data1/jail-DIR/dev rule -s 3 applyset cd /mnt/data1/jail-DIR ln -sf dev/null kernel cp /usr/local/sbin/jkill /mnt/data1/jail-DIR/sbin

jail /mnt/data1/jail-DIR testhostname 192.168.11.100 /bin/sh csh touch /etc/fstab echo 'network_interfaces=""\ hostname="newsystem"\ kern_securelevel_enable="NO"\ sendmail_enable="YES"\ sshd_enable="YES"' > /etc/rc.conf

echo "nameserver 69.55.225.225\ nameserver 69.55.230.3" >> /etc/resolv.conf

vi /etc/crontab remove the adjkerntz lines comment out periodic’s and put this line above them:

1. IF YOU UNCOMMENT THESE, PLEASE ADJUST THEIR RUN TIME

rm -rf /etc/periodic/daily/400.status-disks

check /tmp for crap

vi /etc/periodic/security/100.chksetuid replace: MP=`mount -t ufs | grep -v " nosuid" | awk '{ print $3 }' | sort` with: MP='/' (use single quotes)

mkdir -p /usr/compat/linux/dev

adduser (Add account for user, make sure in wheel group)

Username: user Full name: user Uid (Leave empty for default): Login group [user]: Login group is user. Invite user into other groups? []: wheel Login class [default]: Shell (sh csh tcsh nologin) [sh]: Home directory [/home/user]: Home directory permissions (Leave empty for default): Use password-based authentication? [yes]: Use an empty password? (yes/no) [no]: Use a random password? (yes/no) [no]: y Lock out the account after creation? [no]: Username  : user Password  : <random> Full Name  : user Uid  : 1001 Class  : Groups  : user Home  : /home/user Home Mode  : Shell  : /bin/sh Locked  : no OK? (yes/no): y adduser: INFO: Successfully added (user) to the user database. adduser: INFO: Password for (user) is: 901gmYjO Add another user? (yes/no): n Goodbye!

vi /usr/home/user/.profile (and add to the file): TERM=vt100; export TERM

tzsetup

newaliases

rm /sbin/halt /sbin/reboot ln /sbin/jkill /sbin/halt ln /sbin/jkill /sbin/reboot

1. cd /dev
2. rm console
3. ln -s null console

vi /etc/syslog.conf (comment out console and move to /var/log/messages):

1. • .err;kern.warning;auth.notice;mail.crit /dev/console *.err;kern.warning;auth.notice;mail.crit /var/log/messages

exit exit

cd libexec chflags noschg ld-elf32.so.1 chflags noschg ld-elf.so.1

mv ld-elf32.so.1 ld-elf32.so.1-orig ln ld-elf.so.1 ld-elf32.so.1

chflags schg ld-elf.so.1 chflags schg ld-elf32.so.1

mv /mnt/data1/jail-DIR/usr/sbin/traceroute /mnt/data1/jail-DIR/usr/sbin/_traceroute

echo '#\!/bin/sh\ /usr/sbin/_traceroute -i bce0 $1' >> /mnt/data1/jail-DIR/usr/sbin/traceroute chmod +x /mnt/data1/jail-DIR/usr/sbin/traceroute

cd /usr/ports make -DNOCLEANDEPENDS clean (2450: 15mins , supermicro: 29mins, 2950: 18mins) rm -fr /usr/ports/distfiles/* cp -r /usr/ports /mnt/data1/jail-DIR/usr (2450: 2:00 mins , supermicro: 15mins, 2950: 3mins)

rm /mnt/data1/jail-DIR/root/.history

cd umount /mnt/data1/jail-DIR/dev dump -0a -f /usr/local/jail/template/template /dev/md0 umount /dev/md0 rmdir /mnt/data1/jail-DIR mdconfig -d -u 0

33. setup backups echo '#\!/bin/sh\ backupdir=/data/jail3\ server=backup1\ \

1. 1. ENTRY /etc\
   2. ENTRY /usr/local/etc\
   3. ENTRY /usr/local/jail\
   4. ENTRY /root/logs' > /usr/local/jail/bin/backup.config

on backup1: setup backup dirs: ssh backup1 mkdir -p /data/jail3/0

on backup1, add the system to vi /usr/local/sbin/snapshot_rotate

on mail: vi /usr/local/www/mgmt/cgi/backupgraph.pl (add hostname)

Edit /usr/local/jail/bin/backup.xxx to use the right drives and copy to /usr/local/jail/bin/backup

34. mkdir /root/logs

35. edit sshd_config for security vi /etc/ssh/sshd_config ListenAddress 69.55.229.7 ListenAddress 10.1.2.103

kill -1 `cat /var/run/sshd.pid`

36. add crontab entries crontab -e 5 0 * * * /usr/local/jail/bin/backup.md 1 0 1 * * /usr/local/jail/bin/ipfwreset 0 18 * * * /usr/local/jail/bin/ipfwbackup 4,9,14,19,24,29,34,39,44,49,55,59 * * * * /usr/local/jail/bin/trafstats 0 0,6,12,18 * * * /usr/local/jail/bin/sync_jail_names

• /5 * * * * /usr/local/jail/bin/perc5iraidchk
• /5 * * * * /usr/local/jail/bin/perc4eraidchk

37. Reboot notify script ln -s /usr/local/jail/bin/notify.sh /usr/local/etc/rc.d/notify.sh

38. add to management db (on mail and devweb) jc.ref_machines and jc.ref_templates

uname -r 8.0-RELEASE-p2

insert into ref_machines values (null,'mx2','mx2.johncompanies.com',0,'m'); select machine_id from ref_machines where host='mx2'; +------------+ | machine_id | +------------+ | 35 | +------------+ insert into ref_templates values (,' 8.3-RELEASE-jc2',10,'FreeBSD 8.3',0);

39. add to server/cabinet map. On mail: vi /usr/local/www/mgmt/html/cabinetmap.html

40. add an outside blocking rule to the firewall, so this machine can only be reached from inside the firewall. Follow example already in firewall jail17 is:

00117 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.228.200 00117 deny ip from any to 69.55.228.200

jail19 would be 00119... ipfw add 00109 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 } to 69.55.232.3 ipfw add 00109 deny ip from any to 69.55.232.3

41. select customers for probe map

42. patch jail against starting jails with rtprio

mv /usr/sbin/jail /usr/sbin/jail_ echo '#\!/bin/sh\ /usr/sbin/rtprio -t /usr/sbin/jail_ $*' > /usr/sbin/jail chmod +x /usr/sbin/jail

43. make sure mail works If there are map errors: cd /etc/mail; make maps

44. rdate

cd /usr/ports/sysutils/rdate make install clean

crontab -e 0 0 * * * /usr/local/sbin/rdate -s utcnist.colorado.edu

/usr/local/sbin/rdate -s utcnist.colorado.edu

45. recover space on /usr (optional)

rm -fr /usr/obj

46. wrapper jps

mv /usr/local/sbin/jps /usr/local/sbin/jps_

47. wrapper jls

mv /usr/sbin/jls /usr/sbin/jls_

48. wrapper jexec

mv /usr/sbin/jexec /usr/sbin/jexec_

49. install jtop

cd /usr/ports/sysutils/jtop make install clean

50. block jails from reaching private net echo 'ipfw add 1 deny ip from 69.55.224.0/20 to 10.1.4.0/24' > /usr/local/etc/rc.d/ipfw.sh chmod 0700 /usr/local/etc/rc.d/ipfw.sh

xx. setup fuse

cd /usr/ports/sysutils/fusefs-kmod/ make install

vi /etc/rc.conf fusefs_enable="YES"

sysctl vfs.usermount=1

cd /usr/ports/sysutils/fusefs-sshfs make install

sshfs 1005@usw-s009.rsync.net: /mnt/data1/69.55.234.68-col00001-DIR/mnt