Editing
Jail Server Install
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
= FreeBSD 7.2 = Last updated 2009-06-18 <pre>All time extimates below assume disks aren’t scrubbing. Setup instructions below are for LSI card: 1. make sure bios is setup for bios console redirect 2950: Console redirection: LCD string.. Date to GMT 2. assuming mirrors (or at least disks) created (if not, refer to this), boot to disk 1 of 7.2 skip kernel config (enter) custom install partition -> move cursor to mfid0, hit space (takes you to partition map screen) a for entire disk q to quit and save standard mbr (no boot manager) space to unselect mfid0 cursor over mfid1 space a for entire disk q to quit and save none (leave untouched) cursor over mfid0 space (takes you into part. Screen again) q to exit none Make sure both are checked and tab to ok Label -> Make sure mfid0 is highlighted / 256M swap 2G (for 2950 make it 4G) /var 256M (4.6G) /tmp 256M /usr 4G /mnt/data1 remaining space (no need to newfs) Make sure to toggle S for soft updates on all (should look like UFS2+S Y under the Newfs column) move cursor to mfid1 swap 8G (or 4G if there’s a 3rd drive) /mnt/data2 remaining space (no need to newfs) q to save and exit distributions -> developer yes to install ports exit media -> cd commit -> yes (2450: 14mins, supermicro: 12mins) yes to "visit general config" -> Set root pwd Add user ‘user’ member group is wheel, set password Set tz Networking->interfaces->Fxp0 -> No IPV6 dhcp=no Set hostname & domain Enable sshd exit... exit install -> yes take the cd out and let the machine reboot 3. double check the date/time 4. edit /etc/make.conf echo "WITHOUT_X11=yes \ KERNCONF=jail8 \ BOOT_COMCONSOLE_SPEED=115200" >> /etc/make.conf 5. add settings to /boot/loader.conf and /boot.config echo "-Dh" >> /boot.config echo 'console="comconsole,vidconsole" \ boot_multicons="YES" \ boot_serial="YES" \ mfi_linux_load="YES" \ comconsole_speed="115200"' >> /boot/loader.conf 6. turn off all ttyv's except 0 and 1 in /etc/ttys also turn on ttyd0, change type to vt100: vi /etc/ttys ttyv2 "/usr/libexec/getty Pc" cons25 off secure ttyv3 "/usr/libexec/getty Pc" cons25 off secure ttyv4 "/usr/libexec/getty Pc" cons25 off secure ttyv5 "/usr/libexec/getty Pc" cons25 off secure ttyv6 "/usr/libexec/getty Pc" cons25 off secure ttyv7 "/usr/libexec/getty Pc" cons25 off secure ttyd0 "/usr/libexec/getty std.9600" vt100 on secure kill -1 1 on console server: vi /etc/remote (rename port to jail18 depending on where and which digi plugged into) test serial console 7. populate hosts echo "10.1.4.3 backup2" >> /etc/hosts echo "10.1.4.8 backup1" >> /etc/hosts 8. put key in authorized_keys on backup2 cd ssh-keygen -t dsa -b 1024 (default location, leave password blank) cat /root/.ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys' cat /root/.ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys' confirm that you can ssh to backup2 and backup1 without getting a login prompt ssh backup2 hostname ssh backup1 hostname 9. create & populate binaries/scripts dirs mkdir -p /usr/local/jail/bin mkdir -p /usr/local/jail/rc.d mkdir -p /usr/local/jail/template/ mkdir /mnt/data1 mkdir /mnt/data2 scp backup2:"/mnt/data4/bin/freebsd7.x/*" /usr/local/jail/bin cd /usr/local/jail/rc.d/ touch quad1 touch deprecated chmod +x * cd /usr/local/jail/bin ln -s /usr/local/jail/rc.d/quad1 quad1 ln -s /usr/local/jail/bin/jailmake_md jailmake ln -s /usr/local/jail/bin/js_md js ln -s /usr/local/jail/bin/canceljail_md canceljail ln -s /usr/local/jail/bin/jailmakeempty_md jailmakeempty ln -s /usr/local/jail/bin/postboot_md postboot rehash 10. edit root's path and login script: vi /root/.cshrc Change alias entries (add G): alias la ls -aG alias lf ls -FAG alias ll ls -lAG alias ls ls -AG alias mbm mb mount alias mbu mb umount add to path: /usr/local/jail/bin (if adaptec card installed, also add /compat/linux/usr/sbin) and alter the prompt, set the following: set prompt = "`/bin/hostname -s` %/# " at the bottom of the file add: set sshtty=`who am i|awk '{print $2}'` /usr/sbin/rtprio 3 -`psj | grep $sshtty | awk '{print $2}'` set shortty=`who am i | awk '{print $2}' | sed -E 's/.*(..)$/\1/'` foreach x (`psj | grep sh | grep $shortty | awk '{print $2}'`) /usr/sbin/rtprio 2 -$x end To load the new file: source /root/.cshrc 11. install cvsup cd /usr/ports/net/cvsup-without-gui make install clean; rehash; mail -s 'cvs installed' dave.boodman@vtext.com < /dev/null (stay close for gettext options, 2450: 27mins, supermicro: 17mins, 2950: 22mins) 12. get latest sources for this release: cd /usr/src echo "*default host=cvsup4.freebsd.org\ *default base=/usr\ *default prefix=/usr\ *default release=cvs tag=RELENG_7_2\ *default delete use-rel-suffix\ *default compress\ src-all" > sup -OR- echo "*default host=cvsup4.freebsd.org\ *default base=/usr\ *default prefix=/usr\ *default release=cvs tag=RELENG_7\ *default delete use-rel-suffix\ *default compress\ src-all" > sup (stable) cvsup sup ; mail -s 'cvs sup done' dave.boodman@vtext.com < /dev/null (2450, ~12mins, supermicro, 27mins, 2950: 7mins) 13. configure new kernel. cd /usr/src/sys/amd64/conf scp backup2:/mnt/data4/build/freebsd/kern_config-7.1-amd64 ./jail8 edit the kernel config and change ident to be the name of the jail: vi jail8 ident jail8 edit /sys/conf/newvers.sh to add –jc2 to the end of the BRANCH string (RELEASE-jc2) vi /sys/conf/newvers.sh notes: http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html 14. bring over patches from backup2 The various patches are in /mnt/data4/build/freebsd/patches on backup2. There are dirs for each version. Not all dirs are populated, but patches for later versions work on older ones unless there is a new patch in the older dir. cd /usr/src scp backup2:"/mnt/data4/build/freebsd/patches/7.2/*" . Apply patches: patch -l < jls-patch 15. build, install kernel and world cd /boot mv kernel kernel.GENERIC cd kernel.GENERIC mkdir hold mv mfi_linux.ko hold/ mv linux.ko hold/ mv linprocfs.ko hold/ mv linsysfs.ko hold/ mv geom_vinum.ko hold/ mv geom_concat.ko hold/ rm *.ko rm *.symbols mv hold/* . rmdir hold/ cd /usr/src make buildkernel installkernel make buildworld ; mail -s 'buildworld done' dave.boodman@vtext.com < /dev/null (2450: 1:56min, supermicro: 59mins, 2950: 38mins) make installworld (2450: 3min, supermicro: 1min, 2950: :34) mergemaster -i delete /var/tmp/temproot delete bsnmpd delete temporary ./etc/hosts delete temporary ./etc/motd delete /var/tmp/temproot cd /sys/modules/geom/geom_vinum make make install 16. populate devfs ruleset scp backup2:/mnt/data4/build/freebsd/devfs.rules /etc 17. populate /etc/rc.conf with IPs and NFS settings vi /etc/rc.conf kern_securelevel_enable="NO" portmap_enable="NO" sendmail_enable="NO" usbd_enable="YES" xntpd_enable="YES" nfs_client_enable="YES" nfs_reserved_port_only="YES" inetd_enable="YES" inetd_flags="-wW -a 10.1.4.102" devfs_system_ruleset="devfsrules_show_all" #ifconfig_xl0="inet 10.1.4.102 netmask 255.255.255.0" #ifconfig_fxp0="inet 69.55.228.53 netmask 255.255.255.0" #ifconfig_fxp0_alias0="inet 69.55.2xx.xx netmask 255.255.255.0" static_routes="t1 office" route_t1="-net 10.1.5 10.1.4.2" route_office="-net 10.1.6 10.1.4.2" gvinum_enable="YES" fsck_y_enable="YES" background_fsck="NO" rc_mfi_raid_tty_log="YES" 18. make sure sysctls are set and preserved after reboot echo "kern.consmute=0\ kern.ipc.shm_use_phys=1\ kern.ipc.shmall=131070\ kern.ipc.shmmax=134217728\ net.inet.tcp.syncookies=0\ kern.maxfiles=32768\ kern.fallback_elf_brand=3\ kern.maxprocperuid=4000\ security.jail.sysvipc_allowed=1\ security.jail.allow_raw_sockets=1\ security.jail.socket_unixiproute_only=1\ security.jail.chflags_allowed=0\ dev.amr.0.allow_volume_configure=1\ compat.linux.osrelease=2.6.12\ vm.pmap.shpgperproc=300" >> /etc/sysctl.conf 19. mount procfs echo "proc /proc procfs rw 0 0" >> /etc/fstab For Dell 2950/2450: echo "linprocfs /usr/compat/linux/proc linprocfs rw 0 0" >> /etc/fstab For Dell 2950: echo "linsysfs /usr/compat/linux/sys linsysfs rw 0 0" >> /etc/fstab mkdir -p /usr/compat/linux/proc mkdir -p /usr/compat/linux/sys 19. enable noatime option NOT APPLICABLE IF RUNNING GVINUM data1 and data2 should look something like: /dev/amrd0s1g /mnt/data1 ufs rw,noatime 2 2 20. reboot. Confirm new kernel is loaded uname -a Check devfs rules: devfs rule showsets devfs rule -s 3 show 21. update ports: cd /usr/ports echo "*default host=cvsup4.FreeBSD.org\ *default base=/usr\ *default prefix=/usr\ *default release=cvs tag=RELENG_7_2\ *default delete use-rel-suffix\ *default compress\ ports-all tag=." > sup cvsup sup; mail -s 'cvs sup ports done' dave.boodman@vtext.com < /dev/null (2450: 18mins, supermicro: 19mins; 2950: 24mins) 22. Install raid mgmt tool For LSI based cards: install linux_base: cd /usr/ports/emulators/linux_base-fc4 make install clean (2450: 7min, supermicro: 3mins, 2950: 14mins) (for LSI) cd /usr/ports/sysutils/linux-megamgr make install clean cd /usr/ports/sysutils/megarc make install clean (for Perc5/i, 6/i) cd /usr/ports/sysutils/linux-megacli2 make install clean Test: rehash; megacli ldinfo lall a0 For adaptec: On jail18: scp /compat/linux/usr/sbin/aaccli user@10.1.4.102:~ mv ~user/aaccli /compat/linux/usr/sbin/aaccli test out; /compat/linux/usr/sbin/aaccli For 3w9690: scp backup2:/d4/build/3ware/tw_cli-freebsd-x86_64-9.5.0.1.tgz /usr/local/sbin cd /usr/local/sbin xzvf tw_cli-freebsd-x86_64-9.5.0.1.tgz rm tw_cli-freebsd-x86_64-9.5.0.1.tgz test out; ./tw_cli /c0 show allstatus 23. install rsync from ports cd /usr/ports/net/rsync make install clean choose default options 25. configure inetd to respond to mrtg load queries echo "load stream tcp nowait user /usr/local/jail/bin/load.pl load.pl" >> /etc/inetd.conf echo "load 12384/tcp" >> /etc/services 26. install bb client adduser Username: bb Full name: bb Uid (Leave empty for default): 1984 Login group [bb]: Login group is bb. Invite bb into other groups? []: Login class [default]: Shell (sh csh tcsh nologin) [sh]: Home directory [/home/bb]: Use password-based authentication? [yes]: Use an empty password? (yes/no) [no]: Use a random password? (yes/no) [no]: yes Lock out the account after creation? [no]: Username : bb Password : <random> Full Name : bb Uid : 1984 Class : Groups : bb Home : /home/bb Shell : /bin/sh Locked : no OK? (yes/no): yes cd /usr/home/bb scp backup2:/mnt/data4/build/bb/bb-freebsd_linuxcompat.tgz . tar xzf bb-freebsd_linuxcompat.tgz edit /home/bb/bbc1.9e-btf/etc/bb-hosts with something like: echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh \ 10.1.4.108 jail8.johncompanies.com # ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts vi /home/bb/bbc1.9e-btf/ext/openfiles MACHINE="jail8,johncompanies,com" # HAS TO BE IN A,B,C FORM cd /usr/home/bb/bbc1.9e-btf/etc ./bbchkcfg.sh (y to questions) ./bbchkhosts.sh (ignore ssh errors) cd ../.. chown -R bb . su bb cd cd bbc1.9e-btf ./runbb.sh start more BBOUT (look for errors) exit echo 'su - bb -c "cd /home/bb/bbc1.9e-btf; ./runbb.sh start"' > /usr/local/etc/rc.d/bb.sh chmod +x /usr/local/etc/rc.d/bb.sh NOTE: to get bb working on amd, had to copy over bin dir from linux dist 27. configure load mrtg, on mail vi /usr/local/www/mgmt/mrtg/mrtg1.cfg (add new entry to file following existing format) 28. configure bb on mail: vi /usr/home/bb/bbsrc/bb1.9e-btf/etc/bb-hosts 10.1.4.102 jail2.johncompanies.com # ssh su bb cd bbsrc/bb/runbb.sh restart ; exit 29. remove reserve space, enable softupdates (probably already set, so not necessary) NOT APPLICABLE IF USING GVINUM cd umount /mnt/data1 umount /mnt/data2 tunefs -m 0 /mnt/data1 tunefs -m 0 /mnt/data2 mount -a 30. configure ntp echo "server 10.1.4.5" > /etc/ntp.conf /usr/sbin/ntpd -p /var/run/ntpd.pid sleep 2; ntpq -p (confirm it’s able to reach our time server) 32. fwd and reverse lookups on ns1c vr johncompanies.com vi internal.johncompanies.com rndc reload johncompanies.com IN private (edit the PTR too) 35. create the jail template touch /mnt/data1/jail mdconfig -a -t vnode -s 1g -f /mnt/data1/jail -u 0 bsdlabel -rw md0 newfs4x -i 4096 /dev/md0c mkdir /mnt/data1/jail-DIR mount /dev/md0c /mnt/data1/jail-DIR cd /usr/ports/sysutils/jailutils make install clean cd /usr/src make world DESTDIR=/mnt/data1/jail-DIR; pagedave (2450: 2:28mins, supermicro: 55mins, 2950: 1h ) cd etc make distribution DESTDIR=/mnt/data1/jail-DIR mount -t devfs devfs /mnt/data1/jail-DIR/dev devfs -m /mnt/data1/jail-DIR/dev rule -s 3 applyset cd /mnt/data1/jail-DIR ln -sf dev/null kernel cp /usr/local/sbin/jkill /mnt/data1/jail-DIR/sbin jail /mnt/data1/jail-DIR testhostname 192.168.11.100 /bin/sh csh touch /etc/fstab echo 'network_interfaces=""\ hostname="newsystem"\ kern_securelevel_enable="NO"\ sendmail_enable="YES"\ sshd_enable="YES"' > /etc/rc.conf echo "nameserver 69.55.225.225\ nameserver 69.55.230.3" >> /etc/resolv.conf vi /etc/crontab remove the adjkerntz lines comment out periodic’s and put this line above them: # IF YOU UNCOMMENT THESE, PLEASE ADJUST THEIR RUN TIME rm -rf /etc/periodic/daily/400.status-disks check /tmp for crap vi /etc/periodic/security/100.chksetuid replace: MP=`mount -t ufs | grep -v " nosuid" | awk '{ print $3 }' | sort` with: MP='/' (use single quotes) mkdir -p /usr/compat/linux/dev adduser (Add account for user, make sure in wheel group) Username: user Full name: user Uid (Leave empty for default): Login group [user]: Login group is user. Invite user into other groups? []: wheel Login class [default]: Shell (sh csh tcsh nologin) [sh]: Home directory [/home/user]: Home directory permissions (Leave empty for default): Use password-based authentication? [yes]: Use an empty password? (yes/no) [no]: Use a random password? (yes/no) [no]: y Lock out the account after creation? [no]: Username : user Password : <random> Full Name : user Uid : 1001 Class : Groups : user Home : /home/user Home Mode : Shell : /bin/sh Locked : no OK? (yes/no): y adduser: INFO: Successfully added (user) to the user database. adduser: INFO: Password for (user) is: 901gmYjO Add another user? (yes/no): n Goodbye! vi /usr/home/user/.profile (and add to the file): TERM=vt100; export TERM tzsetup newaliases rm /sbin/halt /sbin/reboot ln /sbin/jkill /sbin/halt ln /sbin/jkill /sbin/reboot #cd /dev #rm console #ln -s null console vi /etc/syslog.conf (comment out console and move to /var/log/messages): #*.err;kern.warning;auth.notice;mail.crit /dev/console *.err;kern.warning;auth.notice;mail.crit /var/log/messages exit exit cd libexec chflags noschg ld-elf32.so.1 chflags noschg ld-elf.so.1 mv ld-elf32.so.1 ld-elf32.so.1-orig ln ld-elf.so.1 ld-elf32.so.1 chflags schg ld-elf.so.1 chflags schg ld-elf32.so.1 cd /usr/ports make -DNOCLEANDEPENDS clean (2450: 15mins , supermicro: 29mins, 2950: 18mins) rm -fr /usr/ports/distfiles/* cp -r /usr/ports /mnt/data1/jail-DIR/usr (2450: 2:00 mins , supermicro: 15mins, 2950: 3mins) rm /mnt/data1/jail-DIR/root/.history cd umount /mnt/data1/jail-DIR/dev dump -0a -f /usr/local/jail/template/72template /dev/md0c umount /dev/md0c rmdir /mnt/data1/jail-DIR 36. setup backups echo '#\!/bin/sh\ backupdir=/data/jail2\ server=backup1\ \ ## ENTRY /etc\ ## ENTRY /usr/local/etc\ ## ENTRY /usr/local/jail\ ## ENTRY /root/logs' > /usr/local/jail/bin/backup.config on backup1: setup backup dirs: ssh backup1 mkdir -p /data/jail2/0 add the system to vi /usr/local/sbin/snapshot_rotate on mail: vi /usr/local/www/mgmt/cgi/backupgraph.pl (add hostname) Edit /usr/local/jail/bin/backup.xxx to use the right drives and copy to /usr/local/jail/bin/backup 37. mkdir /root/logs 38. edit sshd_config for security vi /etc/ssh/sshd_config ListenAddress 69.55.228.53 ListenAddress 10.1.4.102 kill -1 `cat /var/run/sshd.pid` 39. add crontab entries crontab -e 5 0 * * * /usr/local/jail/bin/backup.md 1 0 1 * * /usr/local/jail/bin/ipfwreset 0 18 * * * /usr/local/jail/bin/ipfwbackup 4,9,14,19,24,29,34,39,44,49,55,59 * * * * /usr/local/jail/bin/trafstats On 2950 with Perc5/i also add: */5 * * * * /usr/local/jail/bin/perc5iraidchk 40. Reboot notify script ln -s /usr/local/jail/bin/notify.sh /usr/local/etc/rc.d/notify.sh 41. add to management db (on mail and devweb) jc.ref_machines and jc.ref_templates uname -r 6.2-RELEASE-jc1 insert into ref_machines values (null,'mx2','mx2.johncompanies.com',0,'m'); select machine_id from ref_machines where host='mx2'; +------------+ | machine_id | +------------+ | 35 | +------------+ insert into ref_templates values ('',' 7.1-RELEASE-jc2',44,'FreeBSD 7.1',0); 42. add to server/cabinet map. On mail: vi /usr/local/www/mgmt/html/cabinetmap.html 43. add an outside blocking rule to the firewall, so this machine can only be reached from inside the firewall. Follow example already in firewall jail17 is: 00117 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.228.200 00117 deny ip from any to 69.55.228.200 jail19 would be 00119... ipfw add 00107 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 69.55.238.150 } to 69.55.230.108 ipfw add 00107 deny ip from any to 69.55.220.108 44. select customers for probe map 47. patch jail against starting jails with rtprio mv /usr/sbin/jail /usr/sbin/jail_ echo '#\!/bin/sh\ /usr/sbin/rtprio -t /usr/sbin/jail_ $*' > /usr/sbin/jail chmod +x /usr/sbin/jail 48. make sure mail works If there are map errors: cd /etc/mail; make maps 50. ntp doesn’t seem to start from rc so, echo '/usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift' > /usr/local/etc/rc.d/ntp.sh chmod 0700 /usr/local/etc/rc.d/ntp.sh 51. recover space on /usr rm -fr /usr/obj 52. setup fuse cd /usr/ports/sysutils/fusefs-kmod/ make install vi /etc/rc.conf fusefs_enable="YES" sysctl vfs.usermount=1 cd /usr/ports/sysutils/fusefs-sshfs make install sshfs 1005@usw-s009.rsync.net: /mnt/data1/69.55.234.68-col00001-DIR/mnt</pre>
Summary:
Please note that all contributions to JCWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
JCWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main page
Recent changes
Random page
Help about MediaWiki
Tools
What links here
Related changes
Special pages
Page information