New Signups

TODO- review

Open source discount signups

URLS to give to customers: https://secure.johncompanies.com/signup/step1.html?svc=bsd-oss

https://secure.johncompanies.com/signup/step1.html?svc=linux-oss

New Signups

New customers sign up for service on our web based forms at www.johncompanies.com

When a signup occurs an email is sent to support@johncompanies.com and signups get written to two files:

/usr/local/www/jc_pub/data/pending

and

/usr/local/www/jc_pub/data/log/log.pending

The second file is simply a running log of signups, and should not be used for anything. The only time it is handy is if you are editing the pending file, then save it, and are told that the file has changed since you started editing it. This means that someone signed up while you were editing the pending file. What you should do is force the save (thus losing their signup) and then cut and paste the lost signup from /htdocs/colocation/data/log/log.pending.

Lines in the pending file look like this:

2005-11-08;newcastle;Frederick Wilson;;1405 South Adams;Fort Worth;Texas;76104;US;referred by nmrc.org;LM-1;2;Limit 10;1;207.13.31.48;106
vaifan@airmail.net;817-798-8637;on;on;
new;add2;;;replace;replace;preserve;elaine.commadev.com

The fields are defined as follows:

signup date;hostname;last, first;company; address;city;state;zip;country;referred by;package id;bandwidth overage option;if gobut is selected, $ amount to limit is listed here;payment method;source ip;template id;
email;phone #;admin;billing;alt
new/replace system;create new account/add to existing;hostname of server being replaced;new IP pref;update/merge contacts;update/merge owner info;update/preserve traffic overage;

When a customer signs up an email will be sent to support@johncompanies.com containing the log entry in the format above

The process for setting up a new customer is as follows:

1. Go to the pending customers screen (choose Mgmt site -> New Signups). New customers will be at the bottom of the "VPS Signups" and "Colo Signups" sections. If they’ve paid automatically (via PayPal) the word “PAID” will appear in the “pmt status” column. The other thing you’ll see in this column is “paid- not cleared”- this means they paid with eCheck (via PayPal) and the check has not cleared. This normally takes 3 biz. days and we wait till it clears and shows “PAID” before setting up the server. Customers who pay with a credit card will not appear as "PAID" here. In that case, an email will be sent to support@johncompanies.com indicating a payment has been made (or failed). Once the payment has been paid/cleared, click “process….”

Before setting someone up, you need to decide whether the signup is fraudulent. Examples of fraud can be found in /usr/local/www/jc_pub/data/fraud-examples Generally,

• NOT FRAUD = non-anonymous email accounts that match who they are (dan@brockman.com, where the person is Dan Brockman)
• NOT FRAUD = referred by a customer that actually exists or by kuro5hin
• MAYBE FRAUD = referred by ‘web search’ or ‘google’
• MAYBE FRAUD = generic sounding address
• MAYBE FRAUD = IP address is out of country and address is in US (use dnsstuff link on main signups screen to lookup where IP is)
• FRAUD = they don’t type in: referred by and/or hostname and/or give the default answer for bandwidth is stop
• FRAUD = multiple orders each from the same IP and/or using same email address
• FRAUD = no hostname or hacker-sounding hostname: 3v1l0n3 (evilone)

If you’re ever in doubt, call the card owner using the phone number given at signup or call the bank number provided for the card. Never use the email as it’s likely the thief’s and not the cardholder’s.

Almost all the info needed is pre-entered into this screen, but some fields will need attention:

• System: The system is already selected based on the template/OS the customer selected.
• Directory: information provided for you to enter once the system is created. (leave bank for managedcolo)
• Disk: indicates how much disk space the server should have. (leave bank for managedcolo)
• Hostname: self-explanatory.
• veid: (linux systems only) should be filled in with the significant digits of the customer ID, ex: col01340 = veid 1340.
• os: indicates which OS the customer wants (feed to linux vm script)
• ip(s): for linux customers, you should click “ipmap” and scroll down to the first available (green) ip amongst others on the same system. Click on the ip to copy it back to the “ip(s)” field on the form. For FreeBSD customers, the ips available for use are already assigned to the system and should be copied back to this screen once the jail is made (see below). For colo customers, choose an IP appropriate to the data center. i.e. for i2b pick an IP from the 229 block.
• start date: should reflect the day the system was created.
• asset tag: colo only
• password: VPS only
• cabinet: colo only
• ats port: colo @ i2b only
• monitored: should only be checked if the system’s ip/services were supplied to castle to place on the monitor (probes) list.

NOTE: you must use a JavaScript enabled browser to enter new customers otherwise you won’t be given correct options for os templates.

3. create the new system using either jailmake or vm scripts. For posterity, on our older linux systems and older OS's (pre virt17) we used to use a custom script for each OS version: vemakecentos3 vemakedebian40 vemakedebian30 vemakedebian31 vemakefedora2 vemakefedora6 vemakefedora7 vemakesuse100 vemakesuse93 vemakecentos4 vemakedebian31 vemakefedora4 vemakerh9 vemakeubuntu5 vemakefedora vemakefedora5 vemakesuse vemakeubuntu606 vemakeubuntu610 vemakeubuntu704

jailmake and vm both email the new customer their welcome email.

When you are done adding a customer, for both systems: copy back the dir and password (supplied by the VPS make script) into the form. For FreeBSD, copy back the IP which the make script will give back to you.

When you are done filling out all the fields in the pending customer form, click “Activate”. This will create the customer in our database and remove them from the pending list. Nothing is emailed to the customer as a result of this action. If the customer paid via credit card, their info needs to be added manually.

Discussion about choosing IPs ...

When you make a new system, you have to choose what IP to give it. New FreeBSD servers are configured with a set amount of IPs, and as you add new systems to that freebsd server you can use one of the IPs assigned to the host, but not currently assigned to a VPS. You can use the js program to see what IPs are available for assigning to new customers.

However, it is not that simple with the linux systems. The linux servers do not get the IPs of their customer systems bound to the actual machine. That is, even on a fully loaded linux system, if you run `ifconfig -a` from the base machine, you only see one IP - the main IP of that system. Further complicating matters is that linux systems can bind multiple IPs - therefore it is not possible to know that the next new linux system should just have the next IP as the last new one that was created.

So, what we do is, new linux systems are simply assigned a starting IP, and no new machine (freebsd or linux) is assigned a base IP anywhere within 92-128 Ips of that IP. So you have 92-128 IPs to grow with for the linux systems that will live on that machine.

So, you simply start creating systems from that starting point – and additional IPs that those systems need should also be added from that range, and even other linux systems on other virt machines should get new IPs from that block of IPs. To find the block of IPs in use, go thru bash history to find last ip used or assigned to a ve. Then, click the “ipmap” link (or pull up Mgmt. -> Reference -> IP Map). Ip’s not already assigned to a machine will show up as green. In general, we want to use ips which have never been assigned, or were relenquishsed years ago (or at least 30 days ago – IMPORTANT!!). D If a customer has selected an IP package – is paying for extra IPs now, then we’re pretty much obliged to assign all those IPs now. Otherwise, we only ever assign 1 IP.

The Welcome Emails

jailmake and vm, as one of its arguments, asks for an email address. This email address is used to send a welcome email to the new user. However, the welcome email is not inside the jailmake/vemake script, and they are not on the jails themselves either. Here are all the various welcome emails we have:

/usr/local/www/jc_pub/data/welcome-freebsdp (dynamic freebsd welcome email, 7.x 8.x)
/usr/local/www/jc_pub/data/welcome-linux (dynamic linux welcome email)
/usr/local/www/jc_pub/data/welcome-debian (dynamic debian/ubuntu welcome email)
/usr/local/www/jc_pub/data/welcome-fedora (dynamic fedora/centos welcome email)
/usr/local/www/jc_pub/data/welcome-freebsd (original, generic freebsd welcome email)
/usr/local/www/jc_pub/data/welcome-freebsd6 (for freebsd6)
/usr/local/www/jc_pub/data/welcome-freebsd7z (for freebsd7 with zfs DEPRECATED)

on the main johncompanies server.

When the jailmake script is run, it issues the `fetch` (or wget) command to retrieve it, i.e. http://www.johncompanies.com/colocation/data/welcome-freebsdp

and saves it as a temp file, mails it off.

Welcome emails not noted as dynamic above, are emailed with the IP appended at the top to the email address specified. The password to these accounts was/is generic- not very good. Welcome emails noted as dynamic are parsed by the make script to include the IP and a generated, random root password.

IMPORTANT NOTE: when creating an older-OS ve on a virt- this almost never happens (i.e. it’s created using the old-style vemakexxxx command), you must use the support address as the email address as the make script will be pulling down the dynamic welcome email when it expects to see the old, generic format and it won’t look good. When you receive the welcome email in support, format properly and resend to customer.

This means three things:

a) In order for jailmake to work, the johncompanies web server needs to be up and running, and that file needs to exist at that URL

b) If you want to edit the welcome email, you only need to edit it on the web server, in one place

c) If a customer for some reason does not receive the welcome email, then you need to go Mgmt -> Reference -> (file), copy and paste it into an email to them. For the old-generic emails, make sure to add this line to the very top:

IP: (their IP)

If the welcome email does not exist, jailmake will still work, in the sense that it will create the system, but no welcome email will be sent.

Dedicated Server Setup

Before starting the OS install you'll need to know the following pieces of info:

• CID:
• Customer has multiple colos: Y/N
• Asset tag: JC-xxxx
• rack/location:
• Service/Package (and any deviations to B/W, nfs space, IPs, etc) and price:
• RAM in system:
• IPs included in plan:
• OS (32 vs 64bit variant):
• Hostname:
• Disk partitions, including swap space:
• Number of initial IPs to assign:
• Timezone:

Some to most of that info will be provided via the new signup page, assuming the customer ordered the server via our order page. The rest will comes from the sales/build department.

Once the server is built, it should be installed in the rack, booted to the BIOS screen and labeled with the asset tag and the customer's CID (if available/established).

If an IPKVM is not already attached, you will need to ascertain which one is available (look in ~user/kvm, usually open for editing in the p4 screen of the mailbox window).

Before loading the server, there's a couple things to do in the BIOS screen:

1. make sure the date and time are set to UTC
2. make sure the server's power restore action is set to: last state (basically what we're trying to do here is if the server has power pulled, when power is restored it should turn back on- assuming it was on when power was pulled. This is what allows our ATS power cycling to work. If this is not set, when the ATS port is turned off and turned back on, the server may not turn back on.
3. set the boot order to disable network booting or other things that may slow down bootup.

After saving and exiting the BIOS, if the server has a raid card installed, you will see the raid BIOS screen. You should enter this screen and setup a raid mirror, or whatever the customer has requested (if a special request was made sales will let you know). Usually we use a 3ware card, to enter the config screen, press ALT-3. Use spacebar to select the 2 drives, tab to 'create unit'. Use defaults. Create a raid1 mirror (unless customer asks for something else). Do not enable write cache, assuming no battery exists. F8 to save and exit.

Reboot and load the OS- follow the instructions for pulling the ISO into the IPKVM and booting to it here.

The install should be for the server version of whatever OS was requested. We typically do not install anything other than an sshd (and ports (tree) for FreeBSD). We don't install a GUI environment. We do not setup auto updates. We do not encrypt home directories.

When selecting and IP, take into consideration the data center: an install at i2b should use an IP from the IPs routed to i2b, and vice versa. A customer's package may come with multiple IPs, however unless they indicate they want more assigned initially (and show good reason for doing so) we only assign 1 IP. If they are to receive more IPs, you will/may need to configure those post-install (depending on the OS). Ideally all IPs they receive are on the same class C block.

When picking a NIC (most of the installs we do are network versions and require the network to download OS components), it's somewhat of a crap shoot as to which NIC to choose and configure. Usually we try to begin pinging the IP we assign and watch for it to respond immediately after configuring the NIC. If it does not ping, you can:

1. ask the NOC to swap the network cable to the alternate port
2. go back and configure the alternate NIC (you may have to restart the OS install if it doesn't properly unconfigure the initial NIC, or you can't change it's config to alleviate an IP conflict)

We set DNS: 69.55.229.3, 69.55.225.225 (for a server @ i2b), 69.55.225.225, 69.55.230.3 (for a server @ castle)

We setup a regular user account: 'user' and we set that password to 'newrootNNNN' where NNNN is the sig digits of the customer's CID. i.e. for col01233 the password is 'newroot1233'. In FreeBSD, make sure to add 'user' to the wheel group (member of). If a root password is allowed to be set, we use the same password: 'newrootNNNN'

Once the OS is installed and you've rebooted (after disconnecting the ISO from the KVM) you should confirm all is working: swap, ram, disk partitions, network is as it should be. If you were only able to assign 1 DNS server in setup, please add another to /etc/resolv.conf:

@i2b: 69.55.229.3, 69.55.225.225 @castle: 69.55.225.225, 69.55.230.3

Assuming they have a 3ware raid card, you will need to transfer in the raid CLI tool and our handy script so the customer may check on the health of their raid array, as instructed/encouraged by the welcome email they will receive.

This package is located on mail;

fetch http://johncompanies.com/tools/3ware/L64.tgz
tar xvzf L64.tgz

The version you choose will of course depend on the OS/version you're installing. You should copy the tarball to the /usr/local/sbin dir (or something in their path) and untar'd. You should run the script included in the tarball called checkraid.sh. If it gives an error, this is due to the fact that on some servers the raid card is recognized as 'c1' and on others 'c0' or something different. Determine which it is for this server by running:

tw_cli info

which will give you output like:

Ctl   Model        (V)Ports  Drives   Units   NotOpt  RRate   VRate  BBU
------------------------------------------------------------------------
c1    9650SE-8LPML 8         6        1       0       5       1      -

In this example it's 'c1' so edit checkraid.sh and change all instances of 'c0' to 'c1'

If we're adding additional IPs (and were not able to do so during initial install), configure as follows:

Centos/Fedora:

 cd /etc/sysconfig/network-scripts/
 cp -p ifcfg-eth* ~
 fetch http://johncompanies.com/tools/network/centos/ifcfg-eth.tgz
 tar xvzf ifcfg-eth.tgz

 vi ifcfg-eth0:

DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=none
IPADDR=69.55.231.XXX  # change this to main IP address
PREFIX=24
GATEWAY=69.55.231.1   # change this to 229.1 if IP addr in other block
DNS1=69.55.229.3
DNS2=69.55.225.225
DNS3=69.55.230.3
DOMAIN="johncompanies.com"
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth0"

  vi  eth0:*

DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=none
IPADDR=69.55.231.XXX  # change this to IP address
PREFIX=24
IPV4_FAILURE_FATAL=yes
IPV6INIT=no

Repeat above as needed for each IP address

Remove the un-needed files.

rm ifcfg-eth0:[3-8]  # change numbers to fit how many IP addresses are needed

Debian/Ubuntu:

cd /etc/network
vi interfaces

Old file was:
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 69.55.227.4
        netmask 255.255.255.0
        network 69.55.227.0
        broadcast 69.55.227.255
        gateway 69.55.227.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 69.55.225.225
        dns-search yeppernet.com

new file is:
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 69.55.227.44
        netmask 255.255.255.0
        network 69.55.227.0
        broadcast 69.55.227.255
        gateway 69.55.227.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 69.55D.225.225
        dns-search yeppernet.com

auto eth0:0
iface eth0:0 inet static
        address 69.55.227.55
        netmask 255.255.255.0
        network 69.55.227.0
        broadcast 69.55.227.255
        gateway 69.55.227.1

 /etc/init.d/networking restart

(make sure you have console in case you screw that up)

ifconfig
eth0      Link encap:Ethernet  HWaddr 00:30:48:28:9d:50
          inet addr:69.55.227.44  Bcast:69.55.227.255  Mask:255.255.255.0
          inet6 addr: fe80::230:48ff:fe28:9d50/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2850149 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2595566 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:1193068127 (1.1 GB)  TX bytes:2046984105 (1.9 GB)
          Base address:0x3000 Memory:fc400000-fc420000

eth0:0    Link encap:Ethernet  HWaddr 00:30:48:28:9d:50
          inet addr:69.55.227.55  Bcast:69.55.227.255  Mask:255.255.255.0
          inet6 addr: fe80::230:48ff:fe28:9d50/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2850149 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2595566 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:1193068127 (1.1 GB)  TX bytes:2046984105 (1.9 GB)
          Base address:0x3000 Memory:fc400000-fc420000

Ubuntu 18.04

Ubuntu 18.04 uses netplan instead of /etc/network/interfaces.

vi /etc/neplan/50-cloud-init.yaml

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
  version: 2
  renderer: networkd
  ethernets:
    enp0s25:
     dhcp4: no
     addresses: [69.55.229.26/24]
     gateway4: 69.55.229.1  
     nameservers:
       addresses: [69.55.229.3,8.8.8.8,8.8.4.4]

netplan apply

The very last thing you should do before logging off the server is to run:

history -c

in every shell you were in so they don't see what we were doing :)

Assuming this server is connected to an ATS, you'll want to confirm the server is connected to the port you think it is, and test the functionality- that you can power cycle the port and the server will come back up when power is restored. You should do a test power cycle (via the mgmt or AM-based ATS control) while the server is in post or BIOS. Basically, just NOT while the OS is running and disks mounted.

You should confirm the switch ports to which the server is connected (maybe has a 2nd port for the RMM). While the server is rebooting during your ATS test, you can observe output on the switch console that looks like:

.Mar  7 09:47:45 PST: %LINK-3-UPDOWN: Interface FastEthernet0/14, changed state to down
.Mar  7 09:47:46 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/14, changed state to down
.Mar  7 09:47:47 PST: %LINK-3-UPDOWN: Interface FastEthernet0/14, changed state to up
.Mar  7 09:47:48 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/14, changed state to up

This confirms the server is on port 14.

Take this info (ATS, switch ports plus the asset tag) and:

1. activate (or add) the server. Update the cabinet, ATS and port, asset tag. (Switch port not kept in mgmt)
2. update the cabinet map to add the server to the correct location, along with the ATS port, and switch port(s)
3. login to the switch serial console and add the customer's CID (and hostname if they have mult. servers) to the port(s) description tag. Repeat for the RMM port if they have one:

switch-p25#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
switch-p25(config)#int fa0/10
switch-p25(config-if)#des col01233 myhostname
switch-p25(config)#int fa0/11
switch-p25(config-if)#des col01233 myhostname (RMM)
switch-p25(config-if)#end
switch-p25#
.Mar  7 11:37:17 PST: %SYS-5-CONFIG_I: Configured from console by console
switch-p25#wr me
Building configuration...
[OK]
switch-p25#

At this point it's safe to hand the server over to the customer. Use the 'new colo welcome' paste. Edit for:

1. IP(s)
2. if it's FreeBSD use the 'root' & 'user' password line, if it's Ubuntu use/edit the 'root' password line, depending on if you set the root password or not, or what you setup for the normal user account
3. the correct DNS depending on which data center they're at (remove ns3c if the server is at castle)
4. if they have no raid card, remove that section
5. if they have an RMM, include the RMM paste somewhere in the email

If they're already a customer, you can exclude the info about their AM login info (since they have it already). If they're not, you'll need to reset their AM pass to see it.

Lastly, assuming this server was installed for a new customer and the customer was not yet activated, the CID was probably not established and the server has no CID label. You'll need to make a note to label it (and add the hostname too if they have multiple servers) next time you're at the data center, or ask the noc staff to do it for you.

Dedicated Server Install Checklist

o Install OS

o Add JCI - adduser, wheel, visudo

  adduser -u 955 jci
  passwd jci
    ''l4mph0st''
  adduser -u 1000 col0XXX
  passwd col0XXXX           
    ''col0XXXX''          (XXXX is their col0XXXX number)
  vi /etc/group
     ( add "jci,user" to group wheel and/or sudo )
  visudo
     ( enable group wheel to do anything with password )

o Install openssh-clients (if not already installed)

  yum -y install openssh-clients

o Update OS

  yum -y update   (for CentOS)

o checkraid.sh

 As noted above install our RAID disk checks

o config IP addresses

o clear last logins

  cp /dev/null /var/log/wtmp

o clear history

  cp /dev/null ~/.bash_history
  history -c
  shutdown -h now

o For a suspected spammer, add rule to firewall2 to limit output for mail ports (ie.) before rule 50

ipfw add 37 pipe 8 ip from 69.55.231.104/30 to any dst-port 25,465,587
ipfw add 37 pipe 8 ip from 69.55.231.108    to any dst-port 25,465,587

After installed in rack

 check ssh
 label switch port
 update cabinet map
 update mgmt cabinet number

RMM Setup

The RMM must be configured via the server's BIOS screen:

Server management > BMC LAN config:
down to Intel RMM3 LAN config:
IP (static) , mask, g/w

user config:
disable anonymous.
root: set pass to newrootNNNN