Editing
Jail Server Install
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
= FreeBSD 7.0 = Last updated 2008-04-30 <pre> All time extimates below assume disks aren’t scrubbing. Setup instructions below are for LSI card: 1. make sure bios is setup for bios console redirect Supermicro: Console redirection: Com port addr: on-board COM A Baud: 38400 Console type: vt100 Flow control: none Console connection: direct Continue cr after post: on 2450: Make sure running bios A09 Console Redirection: VT100/VT220 2. assuming mirrors (or at least disks) created (if not, refer to this), boot to disk 1 of 5.4 skip kernel config (enter) custom install partition -> move cursor to amrd0, hit space (takes you to partition map screen) a for entire disk q to quit and save standard mbr (no boot manager) space to unselect amrd0 cursor over amrd1 space a for entire disk q to quit and save none (leave untouched) cursor over amrd0 space (takes you into part. Screen again) q to exit none Make sure both are checked and tab to ok Label -> Make sure mfid0 is highlighted / 256M swap 2G (for 2950 make it 4G) /var 256M (4.6G) /tmp 256M /usr 4G /mnt/data1 remaining space (no need to newfs) Make sure to toggle S for soft updates on all (should look like UFS2+S Y under the Newfs column) move cursor to mfid1 swap 8G /mnt/data2 remaining space (no need to newfs) q to save and exit distributions -> developer yes to install ports exit media -> cd commit -> yes (2450: 14mins, supermicro: 12mins) yes to "visit general config" -> Set root pwd Add user ‘user’ member group is wheel, set password Set tz Networking->interfaces->Fxp0 -> No IPV6 dhcp=no Set hostname & domain Enable sshd exit... exit install -> yes take the cd out and let the machine reboot 3. double check the date/time 4. edit /etc/make.conf (only add the console speed line if this is a supermicro capable of outputting BIOS redirect at that speed – 2450’s can only do 9600) echo "WITHOUT_X11=yes \ KERNCONF=jail2 \ BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf 5. add settings to /boot/loader.conf and /boot.config echo "-Dh" >> /boot.config echo 'console="comconsole,vidconsole" \ boot_multicons="YES" \ boot_serial="YES" \ mfi_linux_load="YES" \ comconsole_speed="115200"' >> /boot/loader.conf (leave out the speed and mfi lines for 2450s) 6. turn off all ttyv's except 0 and 1 in /etc/ttys also turn on ttyd0, change type to vt100: vi /etc/ttys ttyd0 "/usr/libexec/getty std.9600" vt100 on secure kill -1 1 on console server: vi /etc/remote (rename port to jail18 depending on where and which digi plugged into) test serial console 7. populate hosts echo "10.1.4.3 backup2" >> /etc/hosts echo "10.1.4.8 backup1" >> /etc/hosts 8. put key in authorized_keys on backup2 cd ssh-keygen -t dsa -b 1024 (default location, leave password blank) cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys' confirm that you can ssh to backup2 and backup1 without getting a login prompt ssh backup2 hostname ssh backup1 hostname 9. create & populate binaries/scripts dirs mkdir -p /usr/local/jail/bin mkdir -p /usr/local/jail/rc.d mkdir -p /usr/local/jail/template/ mkdir /mnt/data1 mkdir /mnt/data2 scp backup2:"/mnt/data4/bin/freebsd7.x/*" /usr/local/jail/bin cd /usr/local/jail/rc.d/ touch quad1 touch quad2 touch quad3 touch quad4 touch safe1 touch safe2 touch safe3 touch safe4 chmod +x * cd /usr/local/jail/bin ln -s /usr/local/jail/rc.d/quad1 quad1 ln -s /usr/local/jail/rc.d/quad2 quad2 ln -s /usr/local/jail/rc.d/quad3 quad3 ln -s /usr/local/jail/rc.d/quad4 quad4 ln -s /usr/local/jail/rc.d/safe1 safe1 ln -s /usr/local/jail/rc.d/safe2 safe2 ln -s /usr/local/jail/rc.d/safe3 safe3 ln -s /usr/local/jail/rc.d/safe4 safe4 ln -s /usr/local/jail/bin/jailmake_zfs jailmake --OR-- ln -s /usr/local/jail/bin/jailmake_geom jailmake ln -s /usr/local/jail/bin/js_zfs js --OR-- ln -s /usr/local/jail/bin/js_geom js rehash 10. edit root's path and login script: vi /root/.cshrc add to path: /usr/local/jail/bin (if adaptec card installed, also add /compat/linux/usr/sbin) Change alias entries (add G): alias la ls -aG alias lf ls -FAG alias ll ls -lAG alias ls ls -AG alias mbm mb mount alias mbu mb umount and alter the prompt, set the following: set prompt = "`/bin/hostname -s` %/# " at the bottom of the file add: set sshtty=`who am i|awk '{print $2}'` /usr/sbin/rtprio 3 -`psj | grep $sshtty | awk '{print $2}'` set shortty=`who am i | awk '{print $2}' | sed -E 's/.*(..)$/\1/'` foreach x (`psj | grep sh | grep $shortty | awk '{print $2}'`) /usr/sbin/rtprio 2 -$x end To load the new file: source /root/.cshrc 11. install cvsup cd /usr/ports/net/cvsup-without-gui make install clean; rehash; mail -s 'cvs installed' dave.boodman@vtext.com < /dev/null (stay close for gettext options, 2450: 27mins, supermicro: 17mins, 2950: 22mins) 12. get latest sources for this release: cd /usr/src echo "*default host=cvsup4.freebsd.org\ *default base=/usr\ *default prefix=/usr\ *default release=cvs tag=RELENG_7\ *default delete use-rel-suffix\ *default compress\ src-all" > sup -OR- echo "*default host=cvsup4.freebsd.org\ *default base=/usr\ *default prefix=/usr\ *default release=cvs tag=RELENG_7_1\ *default delete use-rel-suffix\ *default compress\ src-all" > sup cvsup sup ; mail -s 'cvs sup done' dave.boodman@vtext.com < /dev/null (2450, ~12mins, supermicro, 27mins, 2950: 7mins) 13. configure new kernel. for i386: cd /usr/src/sys/i386/conf scp backup2:/mnt/data4/build/freebsd/kern_config-7.0-PAE ./mx2 -or- for amd64: cd /usr/src/sys/amd64/conf scp backup2:/mnt/data4/build/freebsd/kern_config-7.0-amd64 ./jail2 ------ edit the kernel config and change ident to be the name of the jail: vi jail2 ident jail2 edit /sys/conf/newvers.sh to add –jc2 to the end of the BRANCH string (RELEASE-jc2) vi /sys/conf/newvers.sh notes: http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html 14. bring over patches from backup2 The various patches are in /mnt/data4/build/freebsd/patches on backup2. There are dirs for each version. Not all dirs are populated, but patches for later versions work on older ones unless there is a new patch in the older dir. cd /usr/src scp backup2:"/mnt/data4/build/freebsd/patches/7.x/*" . Apply patches: patch -l < jls-patch 15. build, install kernel and world cd /usr/src make buildworld buildkernel installkernel; mail -s 'kernel build done' dave.boodman@vtext.com < /dev/null (2450: 1:56min, supermicro: 59mins, 2950: 38mins) make installworld (2450: 3min, supermicro: 1min, 2950: :34) mergemaster -i delete /var/tmp/temproot delete bsnmpd delete temporary ./etc/hosts delete temporary ./etc/motd delete /var/tmp/temproot cd /usr/src/sys/modules/zfs make make install 16. populate devfs ruleset scp backup2:/mnt/data4/build/freebsd/devfs.rules /etc 17. populate /etc/rc.conf with IPs and NFS settings vi /etc/rc.conf kern_securelevel_enable="NO" portmap_enable="NO" sendmail_enable="NO" usbd_enable="YES" xntpd_enable="YES" nfs_client_enable="YES" nfs_reserved_port_only="YES" sshd_enable="YES" inetd_enable="YES" inetd_flags="-wW -a 10.1.4.102" devfs_system_ruleset="devfsrules_show_all" hostname="jail2.johncompanies.com" ifconfig_xl0="inet 10.1.4.102 netmask 255.255.255.0" ifconfig_fxp0="inet 69.55.228.53 netmask 255.255.255.0" defaultrouter="69.55.228.1" #ifconfig_fxp0_alias0="inet 69.55.2xx.xx netmask 255.255.255.0" static_routes="t1 office" route_t1="-net 10.1.5 10.1.4.2" route_office="-net 10.1.6 10.1.4.2" zfs_enable="YES" 18. make sure sysctls are set and preserved after reboot echo "kern.consmute=0\ kern.ipc.shm_use_phys=1\ kern.ipc.shmall=65535\ kern.ipc.shmmax=134217728\ net.inet.tcp.syncookies=0\ kern.maxfiles=32768\ kern.fallback_elf_brand=3\ kern.maxprocperuid=4000\ security.jail.sysvipc_allowed=1\ security.jail.allow_raw_sockets=1\ security.jail.socket_unixiproute_only=1\ security.jail.chflags_allowed=0\ compat.linux.osrelease=2.6.12\ kern.maxvnodes=400000" >> /etc/sysctl.conf 19. mount procfs echo "proc /proc procfs rw 0 0" >> /etc/fstab For Dell 2950/2450: echo "linprocfs /usr/compat/linux/proc linprocfs rw 0 0" >> /etc/fstab For Dell 2950: echo "linsysfs /usr/compat/linux/sys linsysfs rw 0 0" >> /etc/fstab 20. reboot. Confirm new kernel is loaded uname –a Check devfs rules: devfs rule showsets devfs rule -s 3 show 21. update ports: cd /usr/ports echo "*default host=cvsup4.FreeBSD.org\ *default base=/usr\ *default prefix=/usr\ *default release=cvs tag=RELENG_7_0\ *default delete use-rel-suffix\ *default compress\ ports-all tag=." > sup cvsup sup; mail -s 'cvs sup ports done' dave.boodman@vtext.com < /dev/null (2450: 18mins, supermicro: 19mins; 2950: 24mins) 22. Install raid mgmt tool For LSI based cards: install linux_base: cd /usr/ports/emulators/linux_base-fc4 make install clean (2450: 7min, supermicro: 3mins, 2950: 14mins) (for LSI) cd /usr/ports/sysutils/linux-megamgr make install clean cd /usr/ports/sysutils/megarc make install clean (for Perc5/i) cd /usr/ports/sysutils/linux-megacli make install clean Test: rehash; megacli ldinfo lall a0 For adaptec: On jail18: scp /compat/linux/usr/sbin/aaccli user@10.1.4.102:~ mv ~user/aaccli /compat/linux/usr/sbin/aaccli test out; /compat/linux/usr/sbin/aaccli For 3w9690: scp backup2:/d4/build/3ware/tw_cli-freebsd-x86_64-9.5.0.1.tgz /usr/local/sbin cd /usr/local/sbin xzvf tw_cli-freebsd-x86_64-9.5.0.1.tgz rm tw_cli-freebsd-x86_64-9.5.0.1.tgz test out; ./tw_cli /c0 show allstatus 23. install rsync from ports cd /usr/ports/net/rsync make install clean choose default options 24. install perl from ports Not necessary if linux_base is installed cd /usr/ports/lang/perl5.8/ make install clean; rehash (supermicro: 5min) 25. configure inetd to respond to mrtg load queries echo "load stream tcp nowait user /usr/local/jail/bin/load.pl load.pl" >> /etc/inetd.conf echo "load 12384/tcp" >> /etc/services 26. install bb client adduser Username: bb Full name: bb Uid (Leave empty for default): 1984 Login group [bb]: Login group is bb. Invite bb into other groups? []: Login class [default]: Shell (sh csh tcsh nologin) [sh]: Home directory [/home/bb]: Use password-based authentication? [yes]: Use an empty password? (yes/no) [no]: Use a random password? (yes/no) [no]: yes Lock out the account after creation? [no]: Username : bb Password : <random> Full Name : bb Uid : 1984 Class : Groups : bb Home : /home/bb Shell : /bin/sh Locked : no OK? (yes/no): yes cd /usr/home/bb scp backup2:/mnt/data4/build/bb/bb-freebsd.tar . tar xf bb-freebsd.tar edit /home/bb/bbc1.9e-btf/etc/bb-hosts with something like: echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \ 10.1.4.108 jail2.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts vi /home/bb/bbc1.9e-btf/ext/openfiles MACHINE="jail2,johncompanies,com" # HAS TO BE IN A,B,C FORM cd /usr/home/bb/bbc1.9e-btf/etc ./bbchkcfg.sh (y to questions) ./bbchkhosts.sh (ignore ssh errors) cd ../.. chown -R bb . su bb cd cd bbc1.9e-btf/src make; make install cd .. ./runbb.sh start more BBOUT (look for errors) exit echo 'su - bb -c "cd /home/bb/bbc1.9e-btf; ./runbb.sh start"' > /usr/local/etc/rc.d/bb.sh chmod +x /usr/local/etc/rc.d/bb.sh NOTE: to get bb working on amd, had to copy over bin dir from linux dist 27. configure load mrtg, on mail vi /usr/local/www/mgmt/mrtg/mrtg1.cfg (add new entry to file following existing format) 28. configure bb on mail: vi /usr/home/bb/bbsrc/bb1.9e-btf/etc/bb-hosts 10.1.4.108 jail2.johncompanies.com # ssh su bb cd bbsrc/bb/runbb.sh restart ; exit 29. remove reserve space, enable softupdates (probably already set, so not necessary) NOT APPLICABLE IF USING GVINUM cd umount /mnt/data1 umount /mnt/data2 tunefs -m 0 /mnt/data1 tunefs -m 0 /mnt/data2 mount -a 30. configure ntp echo "server 10.1.4.5" > /etc/ntp.conf /usr/sbin/ntpd -p /var/run/ntpd.pid sleep 2; ntpq -p (confirm it’s able to reach our time server) 31. mrtg switch graphs 32. fwd and reverse lookups on ns1c vr johncompanies.com vi internal.johncompanies.com rndc reload johncompanies.com IN private (edit the PTR too) 33. create zpools Make a g partition: bsdlabel -e /dev/mfid0s1 given: # /dev/aacd0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 262144 0 4.2BSD 2048 16384 16392 b: 4194304 262144 swap c: 143363997 0 unused 0 0 # "raw" part, don't edit d: 524288 4456448 4.2BSD 2048 16384 32776 e: 524288 4980736 4.2BSD 2048 16384 32776 f: 6291456 5505024 4.2BSD 2048 16384 28552 new offset = 6291456 + 5505024 = 11796480 new size is size for 'c' partition minus the new start from above 143363997 - 11796480 = 131567517 So: g: 131567517 11796480 unused 0 0 bsdlabel -e /dev/mfid1s1 change d to g zpool create pool1 mfid0s1g zpool create pool2 mfid1s1g zfs set atime=off pool1 zfs set atime=off pool2 echo ' vfs.zfs.prefetch_disable="1" \ vfs.zfs.arc_min=16777216 \ vfs.zfs.arc_max=33554432 \ vm.kmem_size_max="1G" # for i386\ vm.kmem_size="1G" # for i386\ vm.kmem_size_max="1.5G" # for amd64\ vm.kmem_size="1.5G" # for amd64\ vfs.zfs.zil_disable="1" ' >> /boot/loader.conf 35. create the jail template zfs create -o mountpoint=/mnt/data1/jail -o quota=4G pool1/jail cd /usr/ports/sysutils/jailutils make install clean cd /usr/src make world DESTDIR=/mnt/data1/jail (2450: 2:28mins, supermicro: 55mins, 2950: 1h ) cd etc make distribution DESTDIR=/mnt/data1/jail mount -t devfs devfs /mnt/data1/jail/dev devfs -m /mnt/data1/jail/dev rule -s 3 applyset cd /mnt/data1/jail ln -sf dev/null kernel cp /usr/local/sbin/jkill /mnt/data1/jail/sbin jail /mnt/data1/jail testhostname 192.168.11.100 /bin/sh csh touch /etc/fstab echo 'network_interfaces=""\ hostname="newsystem"\ kern_securelevel_enable="NO"\ sendmail_enable="YES"\ sshd_enable="YES"' > /etc/rc.conf echo "nameserver 69.55.225.225\ nameserver 69.55.230.3" >> /etc/resolv.conf vi /etc/crontab remove the adjkerntz lines comment out periodic’s and put this line above them: # IF YOU UNCOMMENT THESE, PLEASE ADJUST THEIR RUN TIME rm -rf /etc/periodic/daily/400.status-disks check /tmp for crap vi /etc/periodic/security/100.chksetuid replace: MP=`mount -t ufs | grep -v " nosuid" | awk '{ print $3 }' | sort` with: MP='/' (use single quotes) mkdir -p /usr/compat/linux/dev adduser (Add account for user, make sure in wheel group) user/root passwd: 8ico2987 Set root password vi /usr/home/user/.profile (and add to the file): TERM=vt100; export TERM tzsetup newaliases rm /sbin/halt /sbin/reboot ln /sbin/jkill /sbin/halt ln /sbin/jkill /sbin/reboot #cd /dev #rm console #ln -s null console vi /etc/syslog.conf (comment out console and move to /var/log/messages): #*.err;kern.warning;auth.notice;mail.crit /dev/console *.err;kern.warning;auth.notice;mail.crit /var/log/messages cd /libexec mv ld-elf32.so.1 ld-elf32.so.1-orig ln ld-elf.so.1 ld-elf32.so.1 exit exit cd /usr/ports make -DNOCLEANDEPENDS clean (2450: 15mins , supermicro: 29mins, 2950: 24mins) rm -fr /usr/ports/distfiles/* cp -r /usr/ports /mnt/data1/jail/usr (2450: 2:00 mins , supermicro: 15mins, 2950: 3mins) rm /mnt/data1/jail/root/.history cd zfs snapshot pool1/jail@now zfs send pool1/jail@now | cat > /usr/local/jail/template/70template zfs destroy pool1/jail@now zfs destroy pool1/jail rmdir /mnt/data1/jail f 36. setup backups echo '#\!/bin/sh\ backupdir=/data/jail2\ server=backup1\ \ ## ENTRY /etc\ ## ENTRY /usr/local/etc\ ## ENTRY /usr/local/jail\ ## ENTRY /root/logs' > /usr/local/jail/bin/backup.config on backup1: setup backup dirs: mkdir -p /data/jail2/0 add the system to vi /usr/local/sbin/snapshot_rotate on mail: vi /usr/local/www/mgmt/cgi/backupgraph.pl (add hostname) Edit /usr/local/jail/bin/backup to use the right drives 37. mkdir /root/logs 38. edit sshd_config for security vi /etc/ssh/sshd_config ListenAddress 69.55.238.233 ListenAddress 10.1.4.108 kill -1 `cat /var/run/sshd.pid` 39. add crontab entries crontab -e 5 0 * * * /usr/local/jail/bin/backup.zfs 1 0 1 * * /usr/local/jail/bin/ipfwreset 0 18 * * * /usr/local/jail/bin/ipfwbackup 4,9,14,19,24,29,34,39,44,49,55,59 * * * * /usr/local/jail/bin/trafstats On 2950 with Perc5/i also add: */5 * * * * /usr/local/jail/bin/perc5iraidchk 40. Reboot notify script ln -s /usr/local/jail/bin/notify.sh /usr/local/etc/rc.d/notify.sh 41. add to management db (on mail and devweb) jc.ref_machines and jc.ref_templates uname -r 6.2-RELEASE-jc1 insert into ref_machines values (null,'mx2','mx2.johncompanies.com',0,'m'); select machine_id from ref_machines where host='mx2'; +------------+ | machine_id | +------------+ | 35 | +------------+ insert into ref_templates values ('',' 7.1-RELEASE-jc2',44,'FreeBSD 7.1',0); 42. add to server/cabinet map. On mail: vi /usr/local/www/mgmt/html/cabinetmap.html 43. add an outside blocking rule to the firewall, so this machine can only be reached from inside the firewall. Follow example already in firewall jail17 is: 00117 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.228.200 00117 deny ip from any to 69.55.228.200 jail19 would be 00119... ipfw add 00107 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.230.108 ipfw add 00107 deny ip from any to 69.55.220.108 44. select customers for probe map 47. patch jail against starting jails with rtprio mv /usr/sbin/jail /usr/sbin/jail_ echo '#\!/bin/sh\ /usr/sbin/rtprio -t /usr/sbin/jail_ $*' > /usr/sbin/jail chmod +x /usr/sbin/jail 48. make sure mail works If there are map errors: cd /etc/mail; make maps 49. move and pare down generic kernel mv kernel.old/ kernel.GENERIC / too full, remove old kernel modules except for: Id Refs Address Size Name 1 14 0xffffffff80100000 ac6c08 kernel 2 1 0xffffffff80bc8000 1128 mfi_linux.ko 3 4 0xffffffff80bca000 39138 linux.ko 4 1 0xffffffffb48f8000 358c linprocfs.ko 5 1 0xffffffffb48fc000 9d3 linsysfs.ko 6 1 0xffffffffb490c000 80ee8 zfs.ko 50. ntp doesn’t seem to start from rc so, echo '/usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift' > /usr/local/etc/rc.d/ntp.sh chmod 0700 /usr/local/etc/rc.d/ntp.sh </pre>
Summary:
Please note that all contributions to JCWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
JCWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main page
Recent changes
Random page
Help about MediaWiki
Tools
What links here
Related changes
Special pages
Page information