Colo Server Setup: Difference between revisions

From JCWiki
Jump to navigation Jump to search
 
(12 intermediate revisions by one other user not shown)
Line 3: Line 3:


* set the date and time to GMT
* set the date and time to GMT
* set the power loss option to ''last state'' (this ensures that if the DC loses power, the server will turn on when power is restored, and more importantly the same will happen when the ATS port is power cycled).
* disable quiet boot
* set the resume on power loss option to ''last state'' (this ensures that if the DC loses power, the server will turn on when power is restored, and more importantly the same will happen when the ATS port is power cycled).
* make sure USB is not disabled (to allow KVM ISO mount/boot)
* make sure USB is not disabled (to allow KVM ISO mount/boot)
* optionally, disable ROM option for NICs so they don't waste time trying to pxe boot
* optionally, disable ROM option for NICs so they don't waste time trying to pxe boot
* disable booting to network
* adjust the boot order to boot first to HD/RAID. Note: if you're booting to KVM ISO you will have to reboot after the ISO is mounted before you see the Belkin device show up as a bootable device (which you can then move up in the boot order)
* adjust the boot order to boot first to HD/RAID. Note: if you're booting to KVM ISO you will have to reboot after the ISO is mounted before you see the Belkin device show up as a bootable device (which you can then move up in the boot order)


Line 19: Line 21:


It'll be a crapshoot which ethernet device (if there are multiple) is the active one. You basically have to guess and assign an IP to one and immediately after configuring it go ping it. If it's pinging, you guessed right. If not, you can either go back and give the IP to the other nic (this may not work since the other nic may still have the same IP) or you can ask the DC to swap the cat5 cable to the other port.
It'll be a crapshoot which ethernet device (if there are multiple) is the active one. You basically have to guess and assign an IP to one and immediately after configuring it go ping it. If it's pinging, you guessed right. If not, you can either go back and give the IP to the other nic (this may not work since the other nic may still have the same IP) or you can ask the DC to swap the cat5 cable to the other port.
When requested to setup a new user, choose 'user'.
All passwords for user/root are newrootNNNN where the NNNN is the sig. digits from the CID. So if the new CID is col001234 the password is: newroot1234
When setting up Ubu, make sure to set the first (or boot partition) bootable (set bootable flag).
If the customer has not requested that you add more than 1 IP initially (with a valid reason) then just add the 1 IP (or the number they requested if applicable.)


FreeBSD systems will require you to complete the OS load over ftp since the OS won't recognize the IPKVM's ISO as a CD drive (this despite the fact that it clearly booted to it initially). This is why we use "netboot" ISOs.
FreeBSD systems will require you to complete the OS load over ftp since the OS won't recognize the IPKVM's ISO as a CD drive (this despite the fact that it clearly booted to it initially). This is why we use "netboot" ISOs.
After the OS is installed, login and install the proper RAID checking script, if applicable. In most cases it's a 3ware card. The correct file will depend on the distribution:
<pre>
backup2 /d4/build/3ware# ls
9.5.0.1-freebsd64-9690-Upgrade.zip t
9.5.0.1-linux32-9690-Upgrade.zip  tw_cli-freebsd-800x-32_64.tgz
Lib_Utils-1.00-08.noarch.rpm      tw_cli-freebsd-x86_64-9.5.0.1.tgz
MegaCli-8.00.40-1.i386.rpm        tw_cli-linux-x86-9.4.1.tgz
checkraid.sh                      tw_cli-linux-x86-9.5.0.1.tgz
cli_freebsd_10.2.zip              tw_cli-linux-x86-9.5.1.1.tgz
driver-freebsd_6x-9.4.1.3.tgz      tw_cli-linux-x86_64-9.5.3.tgz
new
backup2 /d4/build/3ware#
</pre>
So in the case of linux on 64bit, upload <tt>tw_cli-linux-x86_64-9.5.3.tgz</tt> to their server and unpack in <tt>/usr/lcoal/sbin</tt> then run <tt>checkraid.sh</tt> once to confirm it's working. If not, edit it to reflect the correct controller #. For exanple, run <tt>tw_cli info</tt> which will give you output like:
<pre>[root@backup1 ~]# tw_cli  info
Ctl  Model        (V)Ports  Drives  Units  NotOpt  RRate  VRate  BBU
------------------------------------------------------------------------
c0    9650SE-8LPML 8        6        1      0      5      1      -
</pre>
So, edit <tt>checkraid.sh</tt> so all instances say <tt>c0</tt> instead of <tt>c4</tt>, for example.
You may need to create <tt>/usr/local/sbin</tt> on some OS's
Don't forget to add to /etc/resolv.conf:
nameserver 69.55.225.225
nameserver 69.55.230.3
= Post install =
* update the system record in mgmt to reflect the correct asset tag, ATS port (if at i2b), cabinet and newly-assigned IP. You can keep the notes or not, but at least keep the disk partition and OS note. Leave the password field blank. If this asset isn't in the database, you may be prompted to add it first.
* update [[Cabinetmap]] with location of server, also label switch port and ATS port
* reboot the server and in the process of posting, power down, then power up the port using ATS controls to confirm the server is assigned to the right ATS port.
* make sure billing is all set
* send welcome email (use "colo welcome" from big pastes page).
* mark KVM as unused in KVM notes screen (A0P4)
* add server colo (and hostname if multiple servers) to description field on cisco port:
Go to the serial console for the switch they're connected to (refer to cabinet map as to which switch is in that cabinet):
<pre>
switch-p26>en
Password:
switch-p26#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
switch-p26(config)#int fa0/11
switch-p26(config-if)#des col02171
switch-p26(config-if)#en
switch-p26#wr me
</pre>
* put a note to add a colo label to the server (or ask DC staff to do it- front and back). Also label hostname if cust has mult servers at DC

Latest revision as of 16:27, 20 December 2012

BIOS[edit]

You will want to adjust the following settings (where supported):

  • set the date and time to GMT
  • disable quiet boot
  • set the resume on power loss option to last state (this ensures that if the DC loses power, the server will turn on when power is restored, and more importantly the same will happen when the ATS port is power cycled).
  • make sure USB is not disabled (to allow KVM ISO mount/boot)
  • optionally, disable ROM option for NICs so they don't waste time trying to pxe boot
  • disable booting to network
  • adjust the boot order to boot first to HD/RAID. Note: if you're booting to KVM ISO you will have to reboot after the ISO is mounted before you see the Belkin device show up as a bootable device (which you can then move up in the boot order)

Install the OS[edit]

Refer to IPKVM for details on how to use it to do the install.

RAID setup (if equipped) After you exit the BIOS screen you will see the RAID (3ware) BIOS screen. To enter the configuration press Alt-3. There you will be able to create a new mirror from the drives provided. To do this, move the cursor to each drive and hit the spacebar to select each drive (should be *'d after selecting) then tab down to the menu at the bottom to Create Array. It should be a RAID1 (mirror) with write cache disabled (unless there is a backup battery). Use default options for all other settings. No need to give it a name. When done setting up the options, press F8 to save and exit.

Do a minimal install of whatever OS the customer has selected. Don't install packages beyond the basics (i.e. openssh)

It'll be a crapshoot which ethernet device (if there are multiple) is the active one. You basically have to guess and assign an IP to one and immediately after configuring it go ping it. If it's pinging, you guessed right. If not, you can either go back and give the IP to the other nic (this may not work since the other nic may still have the same IP) or you can ask the DC to swap the cat5 cable to the other port.

When requested to setup a new user, choose 'user'. All passwords for user/root are newrootNNNN where the NNNN is the sig. digits from the CID. So if the new CID is col001234 the password is: newroot1234

When setting up Ubu, make sure to set the first (or boot partition) bootable (set bootable flag).

If the customer has not requested that you add more than 1 IP initially (with a valid reason) then just add the 1 IP (or the number they requested if applicable.)

FreeBSD systems will require you to complete the OS load over ftp since the OS won't recognize the IPKVM's ISO as a CD drive (this despite the fact that it clearly booted to it initially). This is why we use "netboot" ISOs.

After the OS is installed, login and install the proper RAID checking script, if applicable. In most cases it's a 3ware card. The correct file will depend on the distribution:

backup2 /d4/build/3ware# ls
9.5.0.1-freebsd64-9690-Upgrade.zip t
9.5.0.1-linux32-9690-Upgrade.zip   tw_cli-freebsd-800x-32_64.tgz
Lib_Utils-1.00-08.noarch.rpm       tw_cli-freebsd-x86_64-9.5.0.1.tgz
MegaCli-8.00.40-1.i386.rpm         tw_cli-linux-x86-9.4.1.tgz
checkraid.sh                       tw_cli-linux-x86-9.5.0.1.tgz
cli_freebsd_10.2.zip               tw_cli-linux-x86-9.5.1.1.tgz
driver-freebsd_6x-9.4.1.3.tgz      tw_cli-linux-x86_64-9.5.3.tgz
new
backup2 /d4/build/3ware#

So in the case of linux on 64bit, upload tw_cli-linux-x86_64-9.5.3.tgz to their server and unpack in /usr/lcoal/sbin then run checkraid.sh once to confirm it's working. If not, edit it to reflect the correct controller #. For exanple, run tw_cli info which will give you output like:

[root@backup1 ~]# tw_cli  info

Ctl   Model        (V)Ports  Drives   Units   NotOpt  RRate   VRate  BBU
------------------------------------------------------------------------
c0    9650SE-8LPML 8         6        1       0       5       1      -

So, edit checkraid.sh so all instances say c0 instead of c4, for example.

You may need to create /usr/local/sbin on some OS's

Don't forget to add to /etc/resolv.conf:

nameserver 69.55.225.225
nameserver 69.55.230.3

Post install[edit]

  • update the system record in mgmt to reflect the correct asset tag, ATS port (if at i2b), cabinet and newly-assigned IP. You can keep the notes or not, but at least keep the disk partition and OS note. Leave the password field blank. If this asset isn't in the database, you may be prompted to add it first.
  • update Cabinetmap with location of server, also label switch port and ATS port
  • reboot the server and in the process of posting, power down, then power up the port using ATS controls to confirm the server is assigned to the right ATS port.
  • make sure billing is all set
  • send welcome email (use "colo welcome" from big pastes page).
  • mark KVM as unused in KVM notes screen (A0P4)
  • add server colo (and hostname if multiple servers) to description field on cisco port:

Go to the serial console for the switch they're connected to (refer to cabinet map as to which switch is in that cabinet):

switch-p26>en
Password:
switch-p26#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
switch-p26(config)#int fa0/11
switch-p26(config-if)#des col02171
switch-p26(config-if)#en
switch-p26#wr me


  • put a note to add a colo label to the server (or ask DC staff to do it- front and back). Also label hostname if cust has mult servers at DC