Virtuozzo Server Install
4.7
Older versions
4.0
Last updated 2010-03-11
1. enable post info to console
2950:
Serial comm.: on with cons. Redir via com1
Failsafe: 115200
Redir after boot: enabled
Server
Custom- desel all but base
/root/install.log
2. Install OS
CentOS5.x (get latest from http://mirrors.kernel.org/centos/5.4/isos/i386/)
Set partitions:
Disk 1:
/ 3072M
6144M swap
/vz (the rest)
Disk 2:
6144M Swap
Use grub, install to MBR
No firewall
Disable SElinux
Install only base
3. create ssh key, add swsoft key to authorized_keys
(enter to everything)
cd /root/
ssh-keygen -t dsa
echo "ssh-dss AAAAB3NzaC1kc3MAAACBAIB7XHmpcsa1D/vsKS+mzEvBVo7udew/aRahbjnjywL4YXT7ruWzpGo7R3UA8GKIgVHoQGUbuXrJz5ICtBEXOlbgXVEz7Bu5nXOG2C9i/8pmFtUw6GwACII8uZE7t/Zq5CRj+5zN1Rv7n/4X4RAjzk5BHXoZ62a32FTbS4tPvyPrAAAAFQDiUi7YEiffNzGvB544lQpRdn29TwAAAIBkUchLTBE1+sX11iC80yhNh9EGeCMLdkT8UqHsTSaD0myQPxGALuNY4rO31q+25uKqSRDTXd6ZWXp4bplA6Ls6jl7r7XVR9loJ7WvRMqc4Dv68TZ7h2UBqAhu4det2UF4LHim3DNf1gKn9DzWNkmI/N2FvVVAL4aM3sPunVIvUYgAAAIAJX38INABVgnCEBKqpv5FMLm3k9MgMn54oQFmylBbgM0w4SUeuDj3jLrkJf8dT9w8ByEDb8xviexpWrPsbz2DeThle4xgzcyKrUkt2Wugtmm4DwS0S/68BAA9wXoWD/vZ67GBoCKH7uq8VAjUVcCbJ/7IydljnFAZoJpktdwN7iQ== russup@support.sw-soft.com" >> /root/.ssh/authorized_keys
4. setup access to backup2:
echo "10.1.4.3 backup2" >> /etc/hosts
echo "10.1.4.8 backup1" >> /etc/hosts
# Copy id_dsa.pub to backup2 and add it to the end of ~root/.ssh/authorized_keys:
cat .ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys'
cat .ssh/id_dsa.pub | ssh backup1 'cat - >> /root/.ssh/authorized_keys'
ssh backup2 hostname
ssh backup1 hostname
(confirm you can connect w/o entering a password)
5. add bash backward history search
echo "\"\M-z\": history-search-backward
\"\M-x\": history-search-backward " > /root/.inputrc
echo "export GREP_OPTIONS='--color=auto'" >> ~/.bashrc
echo "\"\e[5~\": history-search-backward" >> ~/.inputrc
echo "\"\e[6~\": history-search-forward" >> ~/.inputrc
6. add to /root/.bashrc, create log dir
vi /root/.bashrc
alias v='vzctl'
alias mbm='mb mount'
alias mbu='mb umount'
export PS1="[\u@\h \w]# "
alias cv='cd /vz/private'
alias cv1='cd /vz1/private'
alias cv2='cd /vz2/private'
source /root/.bashrc
mkdir /root/logs
7. make sure serial console and kernel messages are working
add these 2 lines to the top:
vi /boot/grub/menu.lst
serial --unit=0 --speed= 115200
terminal --timeout=10 serial console
and the kernel line should look like:
kernel /boot/vmlinuz-2.4.20-020stab009.24.777-enterprise ro root=/dev/sda1 console=tty0 console=ttyS0,115200
Remove or comment out these lines:
hiddensplashmenu
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
add to /etc/inittab and /etc/securetty
echo "s0:2345:respawn:/sbin/agetty -L ttyS0 115200 linux" >> /etc/inittab
echo "ttyS0" >> /etc/securetty
kill -1 1
DEPRECATED 8. configuring eth1 for management net:
echo "10.1.5/24 via 10.1.4.2
10.1.6/24 via 10.1.4.2" >> /etc/sysconfig/network-scripts/route-eth1
reboot the machine to make sure that the interface is configured properly.
also verify that the routes are in place like so:
netstat -rn | grep 10.1.4.2
10.1.5.0 10.1.4.2 255.255.255.0 UG 40 0 0 eth1
10.1.6.0 10.1.4.2 255.255.255.0 UG 40 0 0 eth1
9. make sure the hostname on the system is properly set - in the form virt4.johncompanies.com - and make sure it is set properly:
cat /etc/sysconfig/network
10. Check/update fstab
vi /etc/fstab
/etc/fstab entries should look something like (note noatime option):
/dev/sda3 /vz ext3 defaults,noatime 1 2
/dev/sdb2 /vz1 ext3 defaults,noatime 1 2
make sure swap is configured in fstab
Use real devices instead of LABEL
11. Install Virtuozzo 4.0
umount /vz
umount /vz1
mke2fs -i1024 -b1024 -j /dev/sda3 &
mke2fs -i1024 -b1024 -j /dev/sdb2 &
Experimentally ran this for sdc on virt12:
mke2fs -i2048 -b2048 -j /dev/sdb1
mount -a
scp backup2:/d4/vzrpms/iso/vzinstall-linux.bin /vz
cd /vz
chmod 0700 vzinstall-linux.bin
./vzinstall-linux.bin
change wd to /vz
Ignore OS version error
Custom install (don’t install templates)
reboot
vzup2date
(run again. till no more updates offered)
reboot
Run vzup2date (make sure to choose enterprise kernel) and/or install latest kernel:
NOTE: skip over the manual kernel load below if you can get the 16G recognized before running vzup2date- then it will dl the ent kernel
NOTE: go to /vz/vzup2date/virtuozzo/linux/i386/4.0.0/CU-2.6.18-028stab068.3 and install kern there (downloaded by vzup2date)
<DEPREACATED>
ftp downloads.swsoft.com
User:
gdawson
Pass:
qjdK1R
cd /updates/Rel400/20080307
bin
get vzkernel-ent-2.6.18-028stab053.10.i686.rpm
get vzmodules-ent-2.6.18-028stab053.10.i686.rpm
bye
rpm -ivh vzkernel-ent-2.6.18-028stab053.10.i686.rpm \
vzmodules-ent-2.6.18-028stab053.10.i686.rpm
</DEPREACATED>
vi /boot/grub/menu.lst
(set default kernel to new kernel)
12. Install ntp
yum install ntp
rm -f /etc/ntp.conf
# Create a new file with one line:
echo "server 10.1.4.5" > /etc/ntp.conf
# Then set ntpd so it will start at boot:
chkconfig --level 3 ntpd on
service ntpd start
# Make sure it’s talking to the ntp server:
ntpq -p
Correct output should have * or + to the left of the IP:
remote refid st t when poll reach delay offset jitter
==============================================================================
10.1.4.105 clock1.redhat.c 2 u - 64 1 0.370 -706677 0.008
13. copy over binaries/scripts from backup2
scp backup2:/mnt/data4/bin/linux/* /usr/local/sbin/
scp 10.1.4.62:/vzconf/jctmpl* /etc/sysconfig/vz-scripts/
scp 10.1.4.62:/vzconf/ve-slm* /etc/sysconfig/vz-scripts/
14. setup root crontab (alter backup time):
crontab -e
1 1 * * * /usr/local/sbin/backup
*/15 * * * * /usr/local/sbin/vim.watchdog.30 > /dev/null
#56 4,16 * * * root /usr/sbin/vzkeyupdate -c
*/5 * * * * /usr/local/sbin/perc5iraidchk
15. setup backups
echo '#!/bin/sh
backupdir=/data/virt11
server=backup1
## ENTRY /etc
## ENTRY /usr/local/sbin
## ENTRY /root/logs
' > /usr/local/sbin/backup.config
16. setup bb
edit passwd and add the bb user:
echo "bb:x:1984:1984:Big Brother:/home/bb:/bin/bash" >> /etc/passwd
# edit group and add the bb group:
echo "bb:x:1984:" >> /etc/group
# rebuild the password database:
pwconv
# Create bb’s home dir and set ownership:
mkdir /home/bb
chown bb.bb /home/bb
# Go to bb’s home and fetch the client tarball:
cd ~bb
scp backup2:/mnt/data4/build/bb/bb-linux.tar .
tar xf bb-linux.tar
cd bbc1.9e-btf/etc/
echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts
echo "10.1.4.61 virt11.johncompanies.com # ssh" >> /home/bb/bbc1.9e-btf/etc/bb-hosts
chmod +r /var/log/messages
cd ..
su bb
./runbb.sh start
exit
# Add this line to /etc/rc.d/rc.local so bb will start at boot:
echo "su - bb -c \"cd /home/bb/bbc1.9e-btf; ./runbb.sh start\"" >> /etc/rc.d/rc.local
17. Uncomment/add lines with the appropriate IP addresses in /etc/ssh/sshd_config:
vi /etc/ssh/sshd_config
ListenAddress 69.55.238.3
ListenAddress 10.1.4.61
And make sure to uncomment
Protocol 2,1
18. setup backup2 nfs mounts
setup portmap to start with server, and start it:
chkconfig --level 3 portmap on
service portmap start
mb mount
run df to confirm that nfs mounts are mounted
19. make alias for /vzconf
ln -s /etc/sysconfig/vz-scripts /vzconf
DEPRECATED 20. copy over the /etc/sysconfig/vz-scripts/4unlimited template file from the last virt server and put it in /etc/sysconfig/vz-scripts
scp 10.1.4.68:/etc/sysconfig/vz-scripts/4unlimited /etc/sysconfig/vz-scripts/
NOTE: if this system is more powerful (RAM, CPU) then the previous, you will want to create a new one using vzsplit:
cd /etc/sysconfig/vz-scripts/
vzsplit -f 4unlimited
21. edit /etc/sysconfig/vz
vi /etc/sysconfig/vz
enable fastboot:
VZFASTBOOT=yes
Disable auto mount
#VZMOUNTS="/vz"
Enable IPV6
IPV6=yes
add/edit to traffic parameters section:
TRAFFIC_SHAPING=yes
BANDWIDTH="eth0:102400"
TOTALRATE="eth0:1:102400"
RATE="eth0:1:50000"
for 2 nics, uncomment this line:
VE_ROUTE_SRC_DEV="eth0"
Make sure the IPTABLES section looks like:
IPTABLES="ip_tables iptable_filter iptable_mangle ipt_limit ipt_multiport ipt_tos ipt_TOS ipt_REJECT ipt_tcpmss ipt_ttl ipt_LOG ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_conntrack ipt_state ipt_helper iptable_nat ip_nat_ftp ip_nat_irc ipt_REDIRECT ipt_multiport ipt_TCPMSS"
22. disable periodic fsck's, lower the root padding on the /vz and /vz1 filesystems to 100MB
tune2fs -i 0 /dev/sda3
tune2fs -i 0 /dev/sdb2
tune2fs -c -1 /dev/sda3
tune2fs -c -1 /dev/sdb2
tune2fs -r 100000 /dev/sda3
tune2fs -r 100000 /dev/sdb2
23. setup vzagent with pub IP (to allow migrations from 3.x):
vzctl set 1 --ipdel all --ipadd 69.55.230.57 --userpasswd vzagent0:k0zub1k --save
vzagent_ctl restart
24. edit ve config files:
From split of 80 take
CPUUNITS
And put into files in /usr/local/sbin/ve-slm.1024MB.conf et. al.
25. customize the control panel:
vzmlocal 1:1:/vz/private/1:/vz/root/1
cd /vz/private/1/fs/root/var/vzcp/static/vz/skins
for f in `ls`; do scp backup2:/d4/build/vzcp/var/vzcp/static/vz/skins/winxp.new/images/logo_main.gif $f/images/logo_pim.gif; done
1) comment 'Backup' tab in /etc/vzcp/pp/tabs.xml:
<tab-bar id="maintance" obj-type="env.core.vt">
<!-- <tab href="$baseurl/vz/ves/backups/index-vzpp" title="Backups" id="maintance.backups"
tab="backups" tip="tab_env_maintenance" cred="vzcp.env.backups.list"
security-obj-type="ve.vz.vt"/> -->
2) change 'href' parameter if 'Maintentance' item in /etc/vzcp/pp/menu.xml:
<!-- <item id="management.maintenance"
href="$baseurl/vz/ves/backups/index-vzpp?branch=env-home,vzpp" icon="backup"
tip="maintenance_screen"> -->
<item id="management.maintenance" href="/vz/cp/vzdir/infrman/envs/repair?branch=env-home,vzpp" icon="backup" tip="maintenance_screen">
3) /etc/vzcp/pp/dashboard.xml
<!-- <item id="management.maintenanance" href="$baseurl/vz/ves/backups/index-vzpp" title="Maintenance" icon="env_backup" tip="maintenance_screen" cred="vzcp.env.backups.list" security-obj-type="ve.vz.vt">-->
<item id="management.maintenanance" href="$baseurl/vzdir/infrman/envs/repair" title="Maintenance" icon="env_backup" tip="maintenance_screen" cred="vzcp.env.backups.list" security-obj-type="ve.vz.vt">
URL for 'href' parameter can be obtained from tabs.xml.
vi /var/vzcp/xsl/screens/vzdir/monitman/traffic.xsl
<sw:text>Traffic for selected period and class (NOTE: this information is not certified by JohnCompanies and is not used to calculate actual usage or charges)</sw:text>
<DEPREACATED>
/vz/private/1/root/var/vzcp/static/vz/skins/
/vz/private/1/root/etc/vzcp/pp/menu.xml
/vz/private/1/root/etc/vzcp/pp/dashboard.xml
pp/tabs.xml
/vz/private/1/root/etc/vzcp/vzcpcon.conf
eg (2.6.1 or 3.0):
rsync -va -e ssh backup2:/d4/build/vzcp/var/vzcp/static/vz/skins/ /vz/private/1/root/var/vzcp/static/vz/skins/
rsync -va -e ssh backup2:/d4/build/vzcp/etc/vzcp/pp/menu.xml /vz/private/1/root/etc/vzcp/pp/menu.xml
rsync -va -e ssh backup2:/d4/build/vzcp/etc/vzcp/pp/dashboard.xml /vz/private/1/root/etc/vzcp/pp/dashboard.xml
rsync -va -e ssh backup2:/d4/build/vzcp/etc/vzcp/vzcpcon.conf /vz/private/1/root/etc/vzcp/vzcpcon.conf
</DEPREACATED>
24. template install
To install new templates:
vzup2date –z
Copy over ubu 8.04 from virt19
rsync -a -e ssh 10.1.4.69:/vz/template/ubuntu/8.04 /vz/template/ubuntu/8.04
install ubuntu 8.04 templates (get latest from vz)
cd /backup4/vzrpms/distro/ubuntu/8.04
rpm -ivh *
vzpkg update cache ubuntu-8.04-x86
26. add traffic shaping support
Re-define class 0:
vi /etc/sysconfig/vz-scripts/networks_classes
0 69.55.224.0/20
0 10.1.4.0/24
make sure 1 is defined (0.0.0.0) and no other classes are needed
Then start it
/etc/init.d/vz shaperrestart
27. udp dos fix
echo "/usr/local/sbin/iptables.sh" >> /etc/rc.d/rc.local
28. make rc.local start before vz
mv /etc/rc.d/rc3.d/S99local /etc/rc.d/rc3.d/S95local
29. disable unwanted services/daemons
chkconfig --level 3 avahi-daemon off
chkconfig --level 3 bluetooth off
chkconfig --level 3 cpuspeed off
chkconfig --level 3 cups off
chkconfig --level 3 firstboot off
chkconfig --level 3 gpm off
chkconfig --level 3 haldaemon off
chkconfig --level 3 hidd off
chkconfig --level 3 ip6tables off
chkconfig --level 3 iptables off
chkconfig --level 3 irqbalance off
chkconfig --level 3 isdn off
chkconfig --level 3 kudzu off
chkconfig --level 3 lvm2-monitor off
chkconfig --level 3 mcstrans off
chkconfig --level 3 mdmonitor off
chkconfig --level 3 messagebus off
chkconfig --level 3 netfs off
chkconfig --level 3 nfslock off
chkconfig --level 3 pcscd off
chkconfig --level 3 rawdevices off
chkconfig --level 3 readahead_early off
chkconfig --level 3 restorecond off
chkconfig --level 3 rpcgssd off
chkconfig --level 3 rpcidmapd off
chkconfig --level 3 smartd off
chkconfig --level 3 yum-updatesd off
30. load mrtg graphs
yum install xinetd
create /etc/xinetd.d/load with the following contents:
echo "service load
{
disable = no
flags = REUSE
socket_type = stream
wait = no
user = root
server = /usr/local/sbin/load.pl
}" > /etc/xinetd.d/load
# Add this line to /etc/services:
echo "load 12384/tcp" >> /etc/services
31. Tasks to do on backup1:
setup backup dirs:
mkdir -p /data/virt11/0
add the system to
vi /usr/local/sbin/snapshot_rotate
32. Tasks to do on mail:
Update backup graph:
vi /usr/local/www/mgmt/cgi/backupgraph.pl
(add hostname)
add to bbhosts:
vi /usr/home/bb/bbsrc/bb1.9e-btf/etc/bb-hosts
su bb
cd
bbsrc/bb/runbb.sh restart ; exit
add to load graph config:
vi /usr/local/www/mgmt/mrtg/mrtg1.cfg
(add new section for new machine)
33. add to management db (on mail and devweb) jc.ref_machines and jc.ref_templates
insert into ref_machines values (null,'virt19','virt19.johncompanies.com',0,'l');
select * from ref_machines;
insert into systems values ('', 'col00001',30,null,null,100,'virt11.johncompanies.com',null,'2010-03-11',null,null,1,0,null,null,null);
select * from systems where hostname='virt19.johncompanies.com';
update ipmap set cid='col00001', sysid=3772, assign_date='2008-01-07', status=1 where ip='69.55.236.2';
update ipmap set cid='col00001', sysid=3772, assign_date='2008-01-07', status=1 where ip='69.55.230.109';
insert into ref_templates values ('','fedora-core-1/20050725',33,'Fedora Core 1',1);
insert into ref_templates values ('','fedora-core-2/20050725',33,'Fedora Core 2',1);
insert into ref_templates values ('','debian-3.1/20050927',33,'Debian 3.1',1);
34. Make sure that ns1c has proper forward and reverse entries for the new system (both private and public addresses), so that new system emailing will work.
vr johncompanies.com
vi internal.johncompanies.com
rndc reload johncompanies.com IN private
(edit the PTR too)
35. add an outside blocking rule to the firewall, so this machine can only be reached from inside the firewall.
Follow example already in firewall virt19 is:
ipfw add 00149 allow ip from { 69.55.230.2 or 69.55.230.10 or 69.55.225.225 or 62.213.72.101 or 64.131.90.27 or 82.200.65.190 or 91.204.24.0/22 or 194.44.160.178 or 195.214.232.10 or 195.214.233.10 or 208.251.212.1 } to 69.55.236.2
ipfw add 00149 deny ip from any to 69.55.236.2
virt12 would be 00142
36. add to server/cabinet map
37. make /vz1/private
mkdir /vz1/private ; chmod 0755 /vz1/private
mkdir /vz1/root ; chmod 0755 /vz1/root
38. put config files in a place they’ll get backed up
mkdir /vzconf/confs
crontab -e
0 0 * * * rm -f /vzconf/confs/*.conf; cp /vzconf/*.conf /vzconf/confs
39. fix pass/vzpp and localse issue with ubu 8.10
vi /vz/template/ubuntu/8.10/x86/config/os/default/post-install
sed '/^password.*pam_unix.so.*sha512/ s~sha512~md5~' -i etc/pam.d/common-password
localedef -i en_US -c -f UTF-8 en_US.UTF-8
40. add tun device support
echo "modprobe tun" >> /etc/rc.local
3.x
Last updated 2007-04-5
1. enable post info to console
Supermicro:
Console redirection:
Com port addr: on-board COM A
Baud: 38400
Console type: vt100
Flow control: none
Console connection: direct
Continue cr after post: off
2450:
Make sure running bios A09
Console Redirection: VT100/VT220
2950:
Serial comm.: on with cons. Redir via com1
Failsafe: 115200
Redir after boot: enabled
6144
Server
Custom- desel all but base
/root/install.log
2. Install OS
Set partitions:
/ 3072M
Disk 1: Swap 4096M
Disk 2: Swap 4096M
/vz (the rest)
No firewall
Disable SElinux
FC4
Choose a custom install, the “minimal” package
CentOS4 Server CD (only option for 2950)
Choose the “minimal” package
3. create ssh key, add swsoft key to authorized_keys
(enter to everything)
cd /root/
ssh-keygen -t dsa
echo "ssh-dss AAAAB3NzaC1kc3MAAACBAIB7XHmpcsa1D/vsKS+mzEvBVo7udew/aRahbjnjywL4YXT7ruWzpGo7R3UA8GKIgVHoQGUbuXrJz5ICtBEXOlbgXVEz7Bu5nXOG2C9i/8pmFtUw6GwACII8uZE7t/Zq5CRj+5zN1Rv7n/4X4RAjzk5BHXoZ62a32FTbS4tPvyPrAAAAFQDiUi7YEiffNzGvB544lQpRdn29TwAAAIBkUchLTBE1+sX11iC80yhNh9EGeCMLdkT8UqHsTSaD0myQPxGALuNY4rO31q+25uKqSRDTXd6ZWXp4bplA6Ls6jl7r7XVR9loJ7WvRMqc4Dv68TZ7h2UBqAhu4det2UF4LHim3DNf1gKn9DzWNkmI/N2FvVVAL4aM3sPunVIvUYgAAAIAJX38INABVgnCEBKqpv5FMLm3k9MgMn54oQFmylBbgM0w4SUeuDj3jLrkJf8dT9w8ByEDb8xviexpWrPsbz2DeThle4xgzcyKrUkt2Wugtmm4DwS0S/68BAA9wXoWD/vZ67GBoCKH7uq8VAjUVcCbJ/7IydljnFAZoJpktdwN7iQ== russup@support.sw-soft.com" >> /root/.ssh/authorized_keys
4. setup access to backup2:
echo "10.1.4.3 backup2" >> /etc/hosts
Copy id_dsa.pub to backup2 and add it to the end of ~root/.ssh/authorized_keys:
cat .ssh/id_dsa.pub | ssh backup2 'cat - >> /root/.ssh/authorized_keys'
ssh backup2
(confirm you can connect w/o entering a password)
5. add bash backward history search
echo "\"\M-z\": history-search-backward
\"\M-x\": history-search-backward " > /root/.inputrc
echo "export GREP_OPTIONS='--color=auto'" >> ~/.bashrc
echo "\"\e[5~\": history-search-backward" >> ~/.inputrc
echo "\"\e[6~\": history-search-forward" >> ~/.inputrc
6. add to /root/.bashrc, create log dir
vi /root/.bashrc
alias v='vzctl'
alias mbm='mb mount'
alias mbu='mb umount'
export PS1="[\u@\h \w]# "
alias cv='cd /vz/private'
alias cv1='cd /vz1/private'
alias cv2='cd /vz2/private'
mkdir /root/logs
7. make sure serial console and kernel messages are working
grub
add these 2 lines to the top:
vi /boot/grub/menu.lst
serial --unit=0 --speed= 115200
terminal --timeout=10 serial console
and the kernel line should look like:
kernel /boot/vmlinuz-2.4.20-020stab009.24.777-enterprise ro root=/dev/sda1 console=tty0 console=ttyS0,115200
Remove the line:
hiddensplashmenu
lilo
Make sure the append line in /etc/lilo.conf looks like this:
append="console=tty console=ttyS0,38400"
If you edit /etc/lilo.conf don’t forget to run lilo
# lilo
Added linux
Added linux-up
Added linux+virtuozzo *
(note: the last console listed is the one that gets output from the rc scripts.)
add to /etc/inittab and /etc/securetty
echo "s0:2345:respawn:/sbin/agetty -L ttyS0 115200 linux" >> /etc/inittab
echo "ttyS0" >> /etc/securetty
kill -1 1
8. configuring eth1 for management net:
echo "10.1.5/24 via 10.1.4.2
10.1.6/24 via 10.1.4.2" >> /etc/sysconfig/network-scripts/route-eth1
reboot the machine to make sure that the interface is configured properly.
also verify that the routes are in place like so:
netstat -rn | grep 10.1.4.2
10.1.5.0 10.1.4.2 255.255.255.0 UG 40 0 0 eth1
10.1.6.0 10.1.4.2 255.255.255.0 UG 40 0 0 eth1
9. make sure the hostname on the system is properly set - in the form virt4.johncompanies.com - and make sure it is set properly:
cat /etc/sysconfig/network
10. Check/update fstab
vi /etc/fstab
/etc/fstab entries should look something like (note noatime option):
/dev/sda3 /vz ext3 defaults,noatime 1 2
/dev/sdb1 /vz1 ext3 defaults,noatime 1 2
make sure swap is configured in fstab
Use real devices instead of LABEL
11. Install Virtuozzo 3.0
If it wasn’t done at install time:
setenforce 0
umount /vz
umount /vz1
mke2fs -i1024 -b1024 -j /dev/sda3 &
mke2fs -i1024 -b1024 -j /dev/sdb2 &
On CentOS 4:
yum install rpm-build; yum update iproute
mount -a
scp backup2:/d4/vzrpms/iso/Rel300.tar.gz /vz/.
cd /vz
tar xzf Rel300.tar.gz
cd Rel300/HW
./install
Choose Enterprise kernel
Vzagent password: k0zub1k
DON’T reboot
vzup2date
User:
gdawson
Pass:
qjdK1R
(run again. till no more updates offered)
reboot
Run vzup2date (make sure to choose enterprise kernel) and/or install latest kernel:
ftp downloads.swsoft.com
cd /updates/Rel300/20061206
bin
get vzkernel-enterprise-2.6.9-023stab033.9.i686.rpm
get vzmodules-enterprise-2.6.9-023stab033.9.i686.rpm
bye
rpm -ivh vzkernel-enterprise-2.6.9-023stab033.9.i686.rpm \
vzmodules-enterprise-2.6.9-023stab033.9.i686.rpm
vi /boot/grub/menu.lst
(set default kernel to new kernel)
12. Install various RPMs
scp backup2:/mnt/data4/linuxrpms/* /tmp
install them in this order:
12a. http fetcher
rpm -ivh /tmp/http_fetcher-1.1.0-1.i386.rpm
rpm -ivh /tmp/fetch-1.0-1.i386.rpm
(ignore error on CentOS4)
Now the fetch command, which vemake calls to email the new system owner, will work (it downloads the welcome email from www.johncompanies.com
12b. ntp client
sometimes ntp isn’t included in which case:
FCx:
rpm -ivh /tmp/libcap-1.10-22.i386.rpm
rpm -ivh /tmp/compat-readline43-4.3-2.i386.rpm
rpm -ivh /tmp/ntp-4.1.2-5.i386.rpm
CentOS4:
rpm -ivh /tmp/ntp-4.1.2-5.i386.rpm
rm -f /etc/ntp.conf
Create a new file with one line:
echo "server 10.1.4.5" > /etc/ntp.conf
Then set ntpd so it will start at boot:
chkconfig --level 3 ntpd on
service ntpd start
Make sure it’s talking to the ntp server:
ntpq -p
Correct output should have something unfer the refid col:
remote refid st t when poll reach delay offset jitter
==============================================================================
10.1.4.105 clock1.redhat.c 2 u - 64 1 0.370 -706677 0.008
12c. adaptec machines only: install aaccli for raid controller (LSI card util is with bin files):
rpm -ivh /tmp/aacapps-4.1-0.i386.rpm
May have to run:
cd /dev
./MAKEDEV.aac aac0
13. copy over binaries/scripts from backup2
scp backup2:/mnt/data4/bin/linux/* /usr/local/sbin/
14. setup root crontab (alter backup time):
crontab -e
1 1 * * * /usr/local/sbin/backup
*/15 * * * * /usr/local/sbin/vim.watchdog.30 > /dev/null
#56 4,16 * * * root /usr/sbin/vzkeyupdate -c
On Perc5/i machines:
*/5 * * * * /usr/local/sbin/perc5iraidchk
15. setup backups
echo '#!/bin/sh
backupdir=/mnt/data4/virt19
## ENTRY /etc
## ENTRY /usr/local/sbin
## ENTRY /root/logs
' > /usr/local/sbin/backup.config
16. setup bb
edit passwd and add the bb user:
echo "bb:x:1984:1984:Big Brother:/home/bb:/bin/bash" >> /etc/passwd
edit group and add the bb group:
echo "bb:x:1984:" >> /etc/group
rebuild the password database:
pwconv
Create bb’s home dir and set ownership:
mkdir /home/bb
chown bb.bb /home/bb
Go to bb’s home and fetch the client tarball:
cd ~bb
scp backup2:/mnt/data4/build/bb/bb-linux.tar .
tar xf bb-linux.tar
cd bbc1.9e-btf/etc/
echo "10.1.4.5 mail.johncompanies.com # BBPAGER BBNET BBDISPLAY smtp ssh" > /home/bb/bbc1.9e-btf/etc/bb-hosts
echo "10.1.4.69 virt19.johncompanies.com # ssh" >> /home/bb/bbc1.9e-btf/etc/bb-hosts
chmod +r /var/log/messages
cd ..
su bb
./runbb.sh start
exit
Add this line to /etc/rc.d/rc.local so bb will start at boot:
echo "su - bb -c \"cd /home/bb/bbc1.9e-btf; ./runbb.sh start\"" >> /etc/rc.d/rc.local
17. Uncomment/add lines with the appropriate IP addresses in /etc/ssh/sshd_config:
vi /etc/ssh/sshd_config
ListenAddress 69.55.232.162
ListenAddress 10.1.4.66
And make sure to uncomment
Protocol 2,1
18. setup backup2 nfs mounts
setup portmap to start with server, and start it:
chkconfig --level 3 portmap on
service portmap start
mb mount
run df to confirm that nfs mounts are mounted
19. m ake alias for /vzconf
ln -s /etc/sysconfig/vz-scripts /vzconf
20. copy over the /etc/sysconfig/vz-scripts/4unlimited template file from the last virt server and put it in /etc/sysconfig/vz-scripts
scp 10.1.4.68:/etc/sysconfig/vz-scripts/4unlimited /etc/sysconfig/vz-scripts/
NOTE: if this system is more powerful (RAM, CPU) then the previous, you will want to create a new one using vzsplit:
cd /etc/sysconfig/vz-scripts/
vzsplit -f 4unlimited
21. edit /etc/sysconfig/vz
vi /etc/sysconfig/vz
enable fastboot:
VZFASTBOOT=yes
add/edit to traffic parameters section:
TRAFFIC_SHAPING=yes
BANDWIDTH="eth0:102400"
TOTALRATE="eth0:1:102400"
for 2 nics, uncomment this line:
VE_ROUTE_SRC_DEV="eth0"
Make sure the IPTABLES section looks like:
IPTABLES="ip_tables iptable_filter iptable_mangle ipt_limit ipt_multiport ipt_tos ipt_TOS ipt_REJECT ipt_tcpmss ipt_ttl ipt_LOG ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_conntrack ipt_state ipt_helper iptable_nat ip_nat_ftp ip_nat_irc ipt_REDIRECT"
22. disable periodic fsck's, lower the root padding on the /vz and /vz1 filesystems to 100MB
tune2fs -i 0 /dev/sda3
tune2fs -i 0 /dev/sdb2
tune2fs -c -1 /dev/sda3
tune2fs -c -1 /dev/sdb2
tune2fs -r 100000 /dev/sda3
tune2fs -r 100000 /dev/sdb2
23. copy over our versions of the vzpp files:
/vz/private/1/root/var/vzcp/static/vz/skins/
/vz/private/1/root/etc/vzcp/pp/menu.xml
/vz/private/1/root/etc/vzcp/pp/dashboard.xml
/vz/private/1/root/etc/vzcp/vzcpcon.conf
eg (2.6.1 or 3.0):
rsync -va -e ssh backup2:/d4/build/vzcp/var/vzcp/static/vz/skins/ /vz/private/1/root/var/vzcp/static/vz/skins/
rsync -va -e ssh backup2:/d4/build/vzcp/etc/vzcp/pp/menu.xml /vz/private/1/root/etc/vzcp/pp/menu.xml
rsync -va -e ssh backup2:/d4/build/vzcp/etc/vzcp/pp/dashboard.xml /vz/private/1/root/etc/vzcp/pp/dashboard.xml
rsync -va -e ssh backup2:/d4/build/vzcp/etc/vzcp/vzcpcon.conf /vz/private/1/root/etc/vzcp/vzcpcon.conf
To disable backups from scratch:
In svcve edit /etc/vzcp/vzcpcon.conf
In section:
<auth>
<type>VEUser</type>
<user>root</user>
Add into/below:
<cred>
<id>maintenance</id>
</cred>
The following:
<cred>
<id>show</id>
</cred>
<cred>
<id>reinstall</id>
</cred>
<cred>
<id>repair_mode</id>
</cred>
Will finally look like:
<cred>
<id>maintenance</id>
<cred>
<id>show</id>
</cred>
<cred>
<id>reinstall</id>
</cred>
<cred>
<id>repair_mode</id>
</cred>
</cred>
</cred>
24. template install
mb mount
install debian 3.0 templates
cd /backup4/vzrpms/distro/debian/3.0/p1/
for f in `cat /backup4/vzrpms/distro/debian/3.0/install_list`; do rpm -ivh $f; done
install debian 3.1 templates
cd /backup4/vzrpms/distro/debian/3.1/p3/
for f in `ls`; do rpm -ivh $f; done
install redhat templates
cd /backup4/vzrpms/distro/rh9/current/
for f in `cat /backup4/vzrpms/distro/rh9/install_list`; do rpm -ivh $f; done
install fedora1 templates
cd /backup4/vzrpms/distro/fc1/current/
for f in `cat /backup4/vzrpms/distro/fc1/install_list`; do rpm -ivh $f; done
change 'krb5-devel-1.3.4-5' to 'krb5-devel-1.3.3-6' in (applies to pre-p5)
/vz/template/fedora-core-devel-1/conf/fedora-core-devel-1.conf.20050314
install fedora2 templates
cd /backup4/vzrpms/distro/fc2/current/
for f in `cat /backup4/vzrpms/distro/fc2/install_list`; do rpm -ivh $f; done
install suse9.2 templates
cd /backup4/vzrpms/distro/suse/current/
for f in `cat /backup4/vzrpms/distro/suse/install_list`; do rpm -ivh $f; done
install centos templates
cd /backup4/vzrpms/distro/centos/current/
for f in `cat /backup4/vzrpms/distro/centos/install_list`; do rpm -ivh $f; done
install fedora4 templates
cd /backup4/vzrpms/distro/fc4/current/
for f in `cat /backup4/vzrpms/distro/fc2/install_list`; do rpm -ivh $f; done
vzpkgcache
To pull templates from another machine (before hand stop ve 1 and move it’s template so you preserve the latest template files just setup by vzup2date):
rsync -a -e ssh 10.1.4.68:/vz/template/ /vz/template/
Make sure ubuntu source is defined:
UBU_SERVER=http://archive.ubuntu.com
And change FC source:
FC_SERVER=http://rhold.fedoraproject.org
In /etc/vztt/vztt.conf
rsync -a -e ssh 10.1.4.68:/etc/vztt/vztt.conf /etc/vztt/
To update the cache:
vzpkg update cache ubuntu-6.10-x86
25. add tun device support
Add to /etc/modules.conf:
echo "alias char-major-10-200 tun" >> /etc/modules.conf
depmod -a
echo "modprobe tun" >> /etc/rc.local
26. add traffic shaping support
Re-define class 0:
vi /etc/sysconfig/vz-scripts/networks_classes
0 69.55.224.0/20
0 10.1.4.0/24
make sure 1 is defined (0.0.0.0) and no other classes are needed
Then start it
/etc/init.d/vz shaperrestart
27. RAIDmon (only applies to LSI 320 PCI card)
cd /tmp
cp /backup4/build/megaraid/linuxmon30.tgz .
tar xzf linuxmon30.tgz
cd linuxmon30
./install
(say no)
mv –f raidmon.init /etc/init.d/raidmon
service raidmon start
tail -f /var/log/messages
(check for output)
28. udp dos fix
echo "/usr/local/sbin/iptables.sh" >> /etc/rc.d/rc.local
29. make rc.local start before vz
mv /etc/rc.d/rc3.d/S99local /etc/rc.d/rc3.d/S95local
30. disable unwanted services/daemons
On CentOS:
chkconfig --del kudzu
chkconfig --del cpuspeed
This should be taken care of by vz install so probably safe to skip
chkconfig --del kudzu
chkconfig --del cups
chkconfig --del gpm
chkconfig --del isdn
chkconfig --del pcmcia
chkconfig --del apmd
chkconfig --del rhnsd
chkconfig --del mdmonitor
31. load mrtg graphs
On FCx xinetd not installed so:
rpm -ivh /tmp/xinetd-2.3.11-1.7x.i386.rpm
create /etc/xinetd.d/load with the following contents:
echo "service load
{
disable = no
flags = REUSE
socket_type = stream
wait = no
user = root
server = /usr/local/sbin/load.pl
}" > /etc/xinetd.d/load
Add this line to /etc/services:
echo "load 12384/tcp" >> /etc/services
On FCx: if xinetd isn’t running, configure it for startup at boot time:
chkconfig --level 3 xinetd on
service xinetd start
32. Tasks to do on backup2:
setup backup dirs:
mkdir -p /mnt/data4/virt19/0
add the system to
vi /mnt/data4/bin/rsync_houseclean
33. Tasks to do on mail:
Update backup graph:
vi /usr/local/www/mgmt/cgi/backupgraph.pl
(add hostname)
add to bbhosts:
vi /usr/home/bb/bbsrc/bb1.9e-btf/etc/bb-hosts
su bb
cd
bbsrc/bb/runbb.sh restart ; exit
add to load graph config:
vi /usr/local/www/mgmt/mrtg/mrtg1.cfg
(add new section for new machine)
34. add to management db (on mail and devweb) jc.ref_machines and jc.ref_templates
insert into ref_machines values (null,'virt19','virt19.johncompanies.com',0,'l');
select * from ref_machines;
insert into systems values ('', 'col00001',43,null,null,100,'virt19.johncompanies.com',null,'2008-01-07',null,null,1,0,null,null,null);
select * from systems where hostname='virt19.johncompanies.com';
update ipmap set cid='col00001', sysid=3772, assign_date='2008-01-07', status=1 where ip='69.55.236.2';
update ipmap set cid='col00001', sysid=3772, assign_date='2008-01-07', status=1 where ip='69.55.230.109';
insert into ref_templates values ('','fedora-core-1/20050725',33,'Fedora Core 1',1);
insert into ref_templates values ('','fedora-core-2/20050725',33,'Fedora Core 2',1);
insert into ref_templates values ('','debian-3.1/20050927',33,'Debian 3.1',1);
35. Make sure that ns1c has proper forward and reverse entries for the new system (both private and public addresses), so that new system emailing will work.
vr johncompanies.com
vi internal.johncompanies.com
rndc reload johncompanies.com IN private
(edit the PTR too)
36. add an outside blocking rule to the firewall, so this machine can only be reached from inside the firewall.
Follow example already in firewall virt19 is:
ipfw add 00149 allow ip from { 69.55.225.225 or 69.55.230.2 or 69.55.230.10 or 69.55.230.3 or 62.213.72.101 or 64.131.90.27 or 80.89.128.178 or 82.200.65.190 or 84.237.120.254 or 89.251.97.2 or 195.214.232.10 or 195.214.233.10 or 208.251.212.1 or 213.130.28.26 } to 69.55.236.2
ipfw add 00149 deny ip from any to 69.55.236.2
virt12 would be 00142
37. add to server/cabinet map
39. make /vz1/private
mkdir /vz1/private ; chmod 0755 /vz1/private
mkdir /vz1/root ; chmod 0755 /vz1/root