Management System / Public Website / Signup / Account Manager

Overview

These systems are all running under the same apache instance on the mail server.

The main ingredients are: apache 1.3.31, mysql, perl, mod_perl and template toolkit.

Apache is listening on 69.55.230.9 ports 80 and 443

The webserver can be stopped with:

apachectl stop

and must be started with:

apachectl startssl

If you run apachectl start none of the ssl-enabled pages will work.

The database may be stopped and started as follows:

/usr/local/etc/rc.d/mysql-server.sh stop
/usr/local/etc/rc.d/mysql-server.sh start

• webroot: /usr/local/www/
• logs: /var/log/httpd-error.log /var/log/httpd-access.log
• Apache config:

/usr/local/etc/apache/httpd.conf:

-SNIP-

<Directory /usr/local/www>
 Options none
</Directory>


<VirtualHost *>
  SSLDisable
  DocumentRoot /usr/local/www/jc_pub
  ServerName johncompanies.com
  ServerAlias www.johncompanies.com
  ServerAlias newwww.johncompanies.com

  <Directory /usr/local/www/jc_pub>
    Options FollowSymLinks
  </Directory>

  Redirect /collocation http://www.johncompanies.com
  Redirect /colocation http://www.johncompanies.com

  ScriptAlias /cgi-bin/ "/usr/local/www/jc_pub/cgi-bin/"
  <Directory /usr/local/www/jc_pub/cgi-bin>
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
    SetHandler None
  </Directory>

</VirtualHost>

<VirtualHost *>
  ServerName mini.johncompanies.com
  SSLDisable
  DocumentRoot /usr/local/www/mini

  <Directory /usr/local/www/mini>
    Options FollowSymLinks
  </Directory>

</VirtualHost>


<VirtualHost _default_:443>

  DocumentRoot "/usr/local/www"

  DocumentRoot /usr/local/www
  <Directory /usr/local/www>
    Deny from All
  </Directory>
  <Directory /usr/local/www/mgmt>
    Allow from All
  </Directory>
  <Directory /usr/local/www/am>
    Allow from All
  </Directory>
  <Directory /usr/local/www/signup>
    Allow from All
  </Directory>
  <Directory /usr/local/www/images>
    Allow from All
  </Directory>
  <Directory /usr/local/www/media>
    Allow from All
  </Directory>
  <Files favicon.ico>
    Allow from All
  </Files>

  SSLEngine on
  SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL

#  SSLCertificateFile         /usr/local/etc/apache/mail.cert
#  SSLCertificateKeyFile     /usr/local/etc/apache/mail.key
  #SSLCertificateFile         /usr/local/etc/apache/ssl.crt/secure.crt
  SSLCertificateFile         /usr/local/etc/apache/ssl.crt/secure.johncompanies.com.crt
  SSLCertificateChainFile    /usr/local/etc/apache/ssl.crt/gd_bundle.crt
  SSLCertificateKeyFile     /usr/local/etc/apache/ssl.key/secure.key

  PerlModule Apache

  # debug stuff
  PerlWarn On
  PerlTaintCheck Off
  PerlModule Apache::StatINC
  PerlInitHandler Apache::Reload
# CustomLog /usr/local/apache/logs/access_log.mgmt common
# ErrorLog /usr/local/apache/logs/error_log.mgmt

  PerlModule Apache::DBI
#  PerlModule Mgmt
#  PerlModule Auth

  PerlSetEnv MGMT_BASE /usr/local/www/mgmt
  PerlSetEnv COMMON_BASE  /usr/local/www/common
  PerlSetEnv SIGNUP_BASE  /usr/local/www/signup
  PerlSetEnv AM_BASE /usr/local/www/am
  PerlSetEnv MGMT_INDEX /mgmt/index.html
  PerlSetEnv SIGNUP_INDEX /signup/step1.html
  PerlSetEnv AM_INDEX /am/dashboard.html
  PerlSetEnv MGMT_LOG_LEVEL debug
  PerlSetEnv MGMT_LOG_FILE /usr/local/www/mgmt/Log/mgmt.log
  PerlSetEnv SIGNUP_LOG_LEVEL info
  PerlSetEnv SIGNUP_LOG_FILE /usr/local/www/signup/Log/signup.log
  PerlSetEnv AM_LOG_LEVEL debug
  PerlSetEnv AM_LOG_FILE /usr/local/www/am/Log/am.log
  PerlSetEnv PP_AUTH_TOKEN Pe3aLk5GdMblAyyLAv5vNYqipcynWdZJKdw1CmcGcIdOz74ujMrDYIov27i
  PerlSetEnv PP_URL 'https://www.paypal.com/cgi-bin/webscr'
  PerlSetEnv PP_EMAIL 'payments@johncompanies.com'
  PerlSetEnv JC_DOMAIN 'secure.johncompanies.com'
  PerlSetEnv CC_LOG_FILE /usr/local/www/mgmt/Log/cc.log
  PerlSetEnv DEV 0
  PerlRequire /usr/local/www/common/conf/startup.pl

  <Directory "/usr/local/www/mgmt">
    SetHandler perl-script
    PerlHandler Mgmt
    PerlSetVar JCMGMTPath /mgmt
    PerlSetVar JCMGMTLoginScript /mgmt/login.html

    <Files LOGIN>
      AuthType Auth
      AuthName JCMGMT
      SetHandler perl-script
      PerlHandler Auth->login
    </Files>

    <FilesMatch ".*\.html$|.*\.cgi|.*\.png">
      AuthType Auth
      AuthName JCMGMT
      PerlAuthenHandler Auth->authenticate
      PerlAuthzHandler  Auth->authorize
      require valid-user
    </FilesMatch>
  </Directory>

  <Directory "/usr/local/www/am">
    SetHandler perl-script
    PerlHandler AM
    PerlSetVar JCAMPath /am
    PerlSetVar JCAMLoginScript /am/login.html
    #PerlSetVar JCAMExpires +1m

    <Files LOGINAM>
     AuthType AMAuth
     AuthName JCAM
     SetHandler perl-script
     PerlHandler AMAuth->login
    </Files>

    <Files PASSRESET>
     SetHandler perl-script
     PerlHandler AMPassR
    </Files>

    <FilesMatch ".*\.html$|.*\.cgi">
     AuthType AMAuth
     AuthName JCAM
     PerlAuthenHandler AMAuth->authenticate
     PerlAuthzHandler AMAuth->authorize
     require valid-user
    </FilesMatch>
  </Directory>

  <Directory /usr/local/www/mgmt/static>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/am/static>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/mgmt/bwgraphs>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/am/bwgraphs>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/images>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/media>
    SetHandler None
  </Directory>
  <Directory /usr/local/www>
    Options FollowSymlinks
  </Directory>

  ScriptAlias /mgmt/cgi/ "/usr/local/www/mgmt/cgi/"

  <Directory /usr/local/www/mgmt/cgi>
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
    SetHandler None
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
  </Directory>

  <Directory "/usr/local/www/signup">
    SetHandler perl-script
    PerlHandler Signup
  </Directory>

  Alias /mgmt/mrtg "/usr/local/www/mgmt/mrtg/data"
  <Directory /usr/local/www/mgmt/mrtg/data/>
    DirectoryIndex index.cgi
    SetHandler None
    Options ExecCGI
    AddHandler cgi-script .cgi
  </Directory>

  Alias /mgmt/rrd "/usr/local/www/mgmt/mrtg/rrd"
  <Directory /usr/local/www/mgmt/mrtg/rrd/>
    DirectoryIndex index.html
    SetHandler None
  </Directory>

  ScriptAlias /mgmt/bb/cgi-bin/ /usr/home/bb/bbsrc/bb1.9i-btf/web/
  Alias /mgmt/bb "/usr/home/bb/bbsrc/bb1.9i-btf/www"
  <Directory /usr/home/bb/bbsrc/bb1.9i-btf/www/gifs>
    SetHandler None
  </Directory>
  <Directory /usr/home/bb/bbsrc/bb1.9i-btf/web>
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
    PerlSetVar JCMGMTLoginScript /mgmt/login.html
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
  </Directory>
  <Directory /usr/home/bb/bbsrc/bb1.9i-btf/www>
    PerlSetVar JCMGMTLoginScript /mgmt/login.html
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
  </Directory>

  Alias /mgmt/awstatsclasses "/usr/local/www/mgmt/awstats/wwwroot/classes/"
  Alias /mgmt/awstatscss "/usr/local/www/mgmt/awstats/wwwroot/css/"
  Alias /mgmt/awstatsicons "/usr/local/www/mgmt/awstats/wwwroot/icon/"
  ScriptAlias /mgmt/awstats/ "/usr/local/www/mgmt/awstats/wwwroot/cgi-bin/"
  Alias /mgmt/icon/ "/usr/local/www/mgmt/awstats/wwwroot/icon/"

  <Directory "/usr/local/www/mgmt/awstats/wwwroot/icon">
    SetHandler None
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>


  <Directory "/usr/local/www/mgmt/awstats/wwwroot">
    PerlSetVar JCMGMTLoginScript /mgmt/login.html
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
    SetHandler None
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Template Toolkit

http://www.template-toolkit.org/docs/manual/index.html

All our dynamic content pages (i.e. anything but jcpub) makes use of the Template Toolkit (TT) framework to create/display our pages. This package allows us to create customied apache handlers via the use of mod_perl. The result is an easy to deploy, easy to develop, flexible and efficient platform.

All these sites are organized under /usr/local/www and consist/rely on several components:

• common/conf/startup.pl - this contains all the use/require code that brings in all the required libraries and modules required to run our sites and TT. It is run once as apache starts up and applies to all sites.

• common/conf/Lib - this directory contains all common libraries, primarly those dealing with database access and access to individual tables.

• mysql:jc - the main table containing everything for mgmt/am/signup sites is contained in the jc database. All data is accessed via the mysql user jc - this is so we can impose restrictions on what that user may do. That access info is stored in common/Lib/DBI.pm

The exception to that is the traffic database which used to be stored on mail, but has since been exported to run locally on the bwdb server itself. i.e. mgmt/am contact the remote traffic database running elsewhere for that data. That access data is stored in common/Lib/DBI_BWDB.pm

Part of TT's efficiency comes from static and natively compiled code/pages. Those pages are stored in:

• mgmt: /tmp/mgmt_templates/
• am: /tmp/am_templates
• signup: /tmp/signup_templates

You could safely delete all of these and apache will just recreate them - though the directory holding the templates must exist, with the right permissions.

To make editing of pages easier we enable apache modules/directives which direct apache to rebuild these pages when it notices there's a difference (httpd.conf):

 PerlModule Apache::StatINC
 PerlInitHandler Apache::Reload

However, any changes made to anything under Lib requires an immediate apache restart to take effect and to keep the site running- any changes w/o a restart will break the site.

Among the directives in httpd.conf you will see a series of variables which are important as they, among other things, direct the mod_perl handlers where certain data is stored:

This is where all the files for a particular set of pages and unique handler are located:

 PerlSetEnv MGMT_BASE /usr/local/www/mgmt
 PerlSetEnv COMMON_BASE  /usr/local/www/common
 PerlSetEnv SIGNUP_BASE  /usr/local/www/signup
 PerlSetEnv AM_BASE /usr/local/www/am

This is the default page to present, if none is specified:

 PerlSetEnv MGMT_INDEX /mgmt/index.html
 PerlSetEnv SIGNUP_INDEX /signup/step1.html
 PerlSetEnv AM_INDEX /am/dashboard.html

Indicates the verbosity and location of the handler logfile (these log files only contain logging which we do internally from our code- these differ from the apache logs which log simple hits and server errors):

 PerlSetEnv MGMT_LOG_LEVEL debug
 PerlSetEnv MGMT_LOG_FILE /usr/local/www/mgmt/Log/mgmt.log
 PerlSetEnv SIGNUP_LOG_LEVEL info
 PerlSetEnv SIGNUP_LOG_FILE /usr/local/www/signup/Log/signup.log
 PerlSetEnv AM_LOG_LEVEL debug
 PerlSetEnv AM_LOG_FILE /usr/local/www/am/Log/am.log

Variables used for pp API access:

 PerlSetEnv PP_AUTH_TOKEN Pe3aLk5GdMblAyyLAv5vNYqipcynWdZJKdw1CmcGcIdOz74ujMrDYIov27i
 PerlSetEnv PP_URL 'https://www.paypal.com/cgi-bin/webscr'
 PerlSetEnv PP_EMAIL 'payments@johncompanies.com'

Our base url:

 PerlSetEnv JC_DOMAIN 'secure.johncompanies.com'

Our credit card info logging file:

 PerlSetEnv CC_LOG_FILE /usr/local/www/mgmt/Log/cc.log

Indicates we are/not in a dev environment:

 PerlSetEnv DEV 0

Public Website (jcpub)

• domain: http://www.johncompanies.com http://johncompanies.com
• webroot: /usr/local/www/jc_pub

Our public-facing website is all static, standard HTML. We have some light javascripting on some pages, but by in large it's a very WYSIWYG site setup.

Signup (signup)

Setup / organization / notes

• domain: https://secure.johncompanies.com/signup/step1.html
• webroot: /usr/local/www/signup

Directory structure:

• am: contains the main handler Signup.pm and the ipn activity log ipn.log
• signup/Lib: contains the log module SLog.pm
• signup/Log: contains the log signup.log, and cardit card activity log cc.log
• signup/Plugin: contains the plugin module AMP.pm and the form fill in module FillInForm.pm
• signup/html: contains all web pages

Usage - server order

Once a customer leaves our public site and visits any page under https://secure.johncompanies.com/signup/ they are in the signup section of our site. There isn't a strict rule that all these pages are signup-related, for instance there are some pages related to payments that run in the signup namespace. This is for convenience since this codebase doesn't require a login/authentication.

The signup code primarly deals with the ordering process. Customers entering the signup process via https://secure.johncompanies.com/signup/step1.html will have the ability to order any product. Several links throughout our site pass optional parameters to this page to pre-select the product the customer has selected. For instance, a link to order a linux package looks like https://secure.johncompanies.com/signup/step1.html?svc=linux

In step 1 the customer is prompted to select the package/service level, their OS, any backup space (a la carte), ownership info and contact info. All customers must provide a billing and admin (technical) contact- both may be the same person. They may also (optionally) provide an alternate contact for the billing and admin contacts. What this means is if we are unable to reach them (perhaps due to some outage on their server, where their primary email is hosted) this is how we'd reach them. They must provide at least 1 non-alternate billing and admin contact (i.e. if they provide just 1 contact and mark it billing and admin, it cannot also be marked alternate. They would need to add another contact and mark it as alternate.)

To update the packages available or OS choices for colo's, edit:
signup/html/step1.html

When editing the OS templates for VPSs, make sure the values for the input options matches the existing template_id (jc:ref_templates.template_id). Also remember that there are 2 sets of linux and freebsd order options to edit (1 normal and 1 oss discount).

In step 2 the customer is prompted for more info based upon their service selection:

any server:

• hostname
• ips - if the package allows add-on ips they are given the option to purchase more
• bandwidth - they are given the option to purchase additional bandwidth
• bandwidth overage - they are prompted to choose from 3 options for when they use all their bandwidth in a given month: 1. stop traffic, 2. bill for additional usage up to an amount they specify, 3. allow unlimited bandwidth and bill accordingly
• payment method (cc, paypal, echeck (paypal), invoice)
• existing customer - we ask if they are an existing customer so we know if this server order is new or a replacement on an existing account or if it is to be set up under a new account

vps:

• disk space - if the package allows add-on space they are given the option to purchase more

colo:

• time zone to set the server to
• # ips to assign initially, and a reason to assign more
• disk partitioning (this should add up to the amount of the disk included with the system)
• special instructions
• (nfs) backup space - provided the option to purchase extra a la carte

In step 3 they are shown the total price for the selected services and given the choice of billing intervals: 1/6/12 months. Discounts and savings are displayed. They are requested to tell us how they found us here. However, if they indicated they were an existing customer on step 2 and they are replacing or adding to an existing account, they are also prompted for info:

• hostname/ip of the server being replaced or the account to which we are adding the server
• selection of a new, permanent ip or a temporary ip
• how to handle the information provided in the signup in regards to existing data on file (replace/merge)

In step 4 they are shown a summary of the entire order, and prompted to accept our TOS. Once they submit this page, their signup is registered regardless of whether they go on to make payment.

The next step depends on their selected payment option. If they are paying via credit card they are prompted to enter their card info, then shown a payment confirmation screen upon successful payment.

If they are paying via paypal, they're sent off to paypal to setup payment, then returned to our site to complete the payment and shown a payment confirmation screen upon successful payment.

Note on payment- they can reload this page and it will not result in a reorder, in fact they cannot go back and accidentally reorder at all. If the customer is ording a colo no payment is taken, rather we do an authorization or setup their paypal payment, but we don't actually take funds until the server is in service.

Usage - colo quote

Another facility provided by the signup code is our colo quote tool. A customer visiting this page https://secure.johncompanies.com/signup/colo_quote.html is able to request a quote on a customized server. The requests end up in the New Signups section of mgmt. Once responded to, the customer is given a special signup link which leads them into the signup process where they may order the customized server.

In the colo quote tool they are asked to provide:

• processor
• RAM
• raid level
• drive size
• IPs
• bandwidth
• nfs backup space
• OS
• requested delivery date
• OS install option (JC-installed or self-install via IPKVM)
• current customer Y/N
• referral source
• comments/questions

To update the list of OS's or the hardware options, you'll edit: signup/html/colo_quote.html

As long as the field names don't change, this will not break compatibility with the quote build tool in mgmt.

Management System (mgmt)

Setup / organization / notes

• domain: https://secure.johncompanies.com/mgmt
• webroot: /usr/local/www/mgmt

Directory structure:

• mgmt: contains the main handler Mgmt.pm and the authentication (Auth.pm) handler, the bigbrother status file bb.html, the credit card gpg key pubring.gpg
• mgmt/Lib: contains the log module MLog.pm
• mgmt/Log: contains the log am.log, and ats activity log ats.log
• mgmt/Plugin: contains the plugins required for the various mgmt pages
• mgmt/html: contains all web pages
• mgmt/static: contains static, non-interpreted content like graphics and javascripts
• mgmt/awstats: our stats package. Not controlled by TT, just lives here so you have to authenticate in mgmt to see it
• mgmt/bwgraphs: deprecated? b/w graphing libraries and code
• mgmt/mrtg: mrtg graphs. Not controlled by TT, just lives here so you have to authenticate in mgmt to see it

All functions located in mgmt are password-protected. Once you're logged in, as long as you keep your browser open (maintain session cookie) you will not need to login again. Users are authenticated via table: jc.users To add users or change password, update this table.

menu

layout

• Pastes

This link is twofold: first it is a dropdown menu of some frequently-used responses to customer questions. Clicking on any of these items will copy the text of the response into your clipboard- provided you are running in a compatible browser (IE). Clicking on the "Pastes" button itself will open up a new window containing many more pastes. Clicking on the links at the top will also copy in the text, which you may review in the page below.

Updating: mgmt/html/pastes.html

• Load

Opens up a new window showing mrtg-style load figures for all our hardware, and network traffic for several important servers (firewall, backup). Clicking on each graph drills down for more data.
See mrtg

• BigBrother

Opens up a new window to the Big Brother monitoring system.
See BigBrother

• New Signups

Opens up a new window to the order processing page.
See New Signups

• PayFlow

Opens up a new window to the the PayFlow manager. See PayFlow

• Cabinet map

Opens up a new window to the cabinet map in this wiki

• Monitoring

Opens up windows to monitor the following:
ATS - monitor/manage ATSs at i2b. See ATS control
Backups - monitor the size and timing of backups running from our servers to backup1. See backup monitoring
Colo Monitor - a bb instance that only monitors customers who have requested/receive the service. See monitor.johncompanies.com
ATS-n - the web-based interface for the ATS. See ATS
Switches - mrtg pages for our switches
System Counts - display's how many VPSs are on each system (according to the database- may not be 100% accurate)
XX VZPP - VZPP (Virtuozzo Power Panel) for each server. See Virtuozzo Reference

• Payments

Paypal history - takes you to the paypal page where you can do advanced searches based on email/subid/txnid
2Checkout V2 - 2Checkout site
Invoice Pmt Entry - allows easy entry of payments for any customer paying via invoice
Arrears list - shows customers who have a balance due
Pmt notices - shows outstanding payment invoices, primarily for customers paying via paypal
Pfp batch - shows items pending processing via payflow. See pfp_batch
Payment links - deprecated
Payment audit - show(ed) discrepancies between pp sub and billing sub
Packages - shows all our current/past packages. See packages
Orphaned R.pmts - shows all paypal subs in the system which are not associated with an account. These should only/all be from people who signed up for payment fraudulently, and all subs should be in cancelled/disabled state. The only real reason to visit this page is in the case of a customer who is adding a server and who's signed up, you would link his paypal sub to his account here.
IPN Input - allows you to manually input a paypal IPN into our system
Request pmt - broken. ??

• Reference

IPKVM - links to our IPKVMs. See IPKVM
VZ Ticket - submit a trouble ticket for help with VZ. See getting help
IP Map - display's ip allocation information. See ipmap
Acct Mgr - quick link to the Account Manager
Asset DB - our asset database. See asset database
Bandwidth - bandwidth query page. See Reporting
Web stats - awstats page
Domaintools - handy whois resource
CrashLog - tool to log the details of a crash. See crash log
DosLog - tool to log the details of a dos attack. See dos log
URL: Linux VPS - takes you right to the linux vps order page
URL: FreeBSD VPS - takes you right to the linux freebsd order page
URL: Dedicated - takes you right to the dedicated server order page
Notices - takes you to the notices tool. See notices
Templates - template management/tracking. See templates
VZ Reference - VZ reference/knowledge base at the parallels website
VZ Templates - template repository at parallels website. The best method for installing new templates is vzup2date. See adding new templates
Bugzilla - our bugzilla site
redit.com - customer portal at redit. Get realtime bandwidth and cabinet power usage
Manual - old/deprecated
johncompanies.com - our website
Linux welcome - generic welcome email
BSD welcome - freebsd welcome email
Debian welcome - debian/ubuntu welcome email
Spam check - site to check if an ip is blacklisted
DEV mgmt - mgmt site on dev server
DEV am - AM on dev server
DEV jcpub - public site on dev server

• Wiki

This.

editing

The dropdown menu is driven by a couple files:
mgmt/static/HM_Arrays.js
mgmt/static/HM_Loader.js

to change or add menu items, edit HM_Arrays.js
covering every topic would be lengthy but here are some tips:

• in each section, each entry but the last must end with a comma
• each entry is formatted: ["name", action, 1,0,N] where N is 0 that means this is a simple button, when N is 1 it is a dropdown and another menu is attached.
• in the case of the first, top-level group:

["Pastes","JavaScript:win1=window.open('','win1'); win1.location='https://secure.johncompanies.com/mgmt/pastes.html'; void(0)",1,0,1],
["Load","JavaScript:win7=window.open('','win7'); win7.location='https://secure.johncompanies.com/mgmt/mrtg/index.cgi'; void(0)",1,0,0],
["BigBrother","JavaScript:win9=window.open('','win9'); win9.location='https://secure.johncompanies.com/mgmt/bb/'; void(0)",1,0,0],
["New Signups","https://secure.johncompanies.com/mgmt/pending.html",1,0,0],
["PayFlow","JavaScript:winpf=window.open('','winpf'); winpf.location='https://manager.paypal.com/'; void(0)",1,0,0],
["Cabinet map","JavaScript:win25=window.open('','win25'); win25.location='https://secure.johncompanies.com/mgmt/cabinetmap.html'; void(0)",1,0,0],
["Monitoring","",1,0,1],
["Payments","",1,0,1],
["Reference","",1,0,1],
["Wiki","JavaScript:winwiki=window.open('','winwiki'); winwiki.location='https://wiki.johncompanies.com/'; void(0)",1,0,0]
],

You see that Pastes is the first menu item and that it has a submenu. That submenu is defined in the HM_Array2_1 array. Likewise, the 7th menu item, Monitoring, has a submenu defined in HM_Array2_7. And that furter has nested menus, the first item in the Monitoring dropdown has a nested menu in HM_Array2_7_1 , and the 13th item in HM_Array2_7_13

• when setting a window to open a new window (most menu items) it's important to keep track of the window name if you want each link to open a unique window. i.e. JavaScript:win10a=window.open(,'win10a'... make the win10a name is unique.

Customer main view page

Most interaction in mgmt is done on or from the main customer page. This is the main landing page you'll get if you're searching for a customer. From this page you can see and edit:

• systems/services
• ownership info
• bandwidth usage info
• contact info
• billing info

Systems

This table displays the systems and services to which a customer has subscribed. Depending on the type of system, certain data will appear. Namely:

• hostname
• root directory (VPS only)
• OS (VPS only)
• disk space allocated (VPS only)
• CT id (linux VPS only)
• sysid
• IPs
• dates of start/cancel/stop
• package ID/smount+interval/next bill date

Any item black and underlined (this goes for anything in mgmt) is clickable and will copy into your clipboard (IE only). Items in this table with special background coloring indicates:

• grey = cancelled system. To un-stop a system, databases need to be manually edited- so be careful.
• blue = monotired by castle. If you uncheck and save the monitor flag (in the system edit screen), you must also update the tracking sheet (a0, p4). Before setting up a probe with castle, make sure that there are no notes indicating the customer has previously requested not to be monitored ("don't monitor"), or “can't monitor” which means castle just can't monitor them for some reason.
• yellow = pending shutdown (cancel date set)

To setup the system for (future) cancel, click the "cancel" link. You will be prompted to provide a cancel date. This will indicate the date you intend to shut down the system. Usually this is in the future since the customer pre-pays for service. Also, you want to set at least 1 day in the future to allow the customer to get the "Shutdown today" email, clueing them in to the fact that their server is going away. The 2 quick links for 5-days and 10-days will set the date in the future (5 or 10 days), both optimized to allow the customer to receive the shutdown reminder notices. If you want the customer not to receive shutdown reminder notices, you should set that field to no. If you want to un-cancel a system, go into the edit screen and remove/clear the cancel date.

When a system has been shut down (or pulled from the rack in the case of a colo), click the "stop" link and set the date it was stopped. If this is the last active system on the account, it will mark the customer cancelled

To edit aspects of the system click the "edit" link under the "actions" column.

edit

From this screen you may edit most if not all aspects of the system in question.

• machine - pretty simple, what host is the VPS on or is this a (jc owned) managedcolo or (customer owned) colo
• directory - (VPS only) where the VPS is located on the host
• disk - (VPS only) how much they actually have. If you need to set this higher, edit systems.disk
• hostname
• veid - linux (VPS only) aka CT id
• os tempalte - (VPS only) this will auto-populate with the templates/os on the host selected in the machine field. To add templates you need to do so via the Templates page
• password - (VPS only) if this is not set this system was setup with our universal/standard password: p455agfa or 8ico2987
• ats port - (colo only) port and ATS to which this server is connected
• cabinet - (colo only) location of colo
• ips - ips assigned to server and the ability to assign more (space delimited if adding mult). A handy link will open ipmap and clicking ips should copy back the ip to the new ip field.
• dates - start/stop/cancel
• monitored - indicate whether this system is on castle's probe map
• base, disk, ip packages - adjust all aspects of packages. If a field is red that means there's a mismatch between what the customer has and what the package covers. i.e. if they have 50 GB of disk and the package only gives 30, you'd need to add a disk package for 20 GB or adjust the base package to include 50 GB. Anything yellow indicates an altered/custom value. i.e. if the package comes with 6 IPs and you set it to 5, that will be highlighted in yellow. Or if the price was $19 and you make it $17, prices will be yellow'd. Green fields in the Totals row indicate a good matchup for the given attribute (i.e. disk allocated to server is the same as disk provided by package(s)).

If you are moving a customer to a new machine, that may be done here- the old machine will be marked as stopped and show up as an additional row in the view screen. So really that is a stop and add operation behind the scenes in mgmt.

add

This works very similar to edit, and like it implies will add a new system to the customer.

mrtg

• domain: https://secure.johncompanies.com/mgmt/mrtg/index.cgi/ https://secure.johncompanies.com/mgmt/mrtg/switch.cgi?s=switch-p3&path=
• webroot: /usr/local/www/mgmt/mrtg

All configuration is done via *.cfg files. The main load graph is found in mrtg1.cfg All other config files are for various switches. Switch config files are rebuilt out of a cron jobs running on mail. This ensures if we change a port name (desc) that the mrtg we look at has the latest info. So if you want to change port naming, please do it in the switch itself. If you have problems getting new devices setup or change existing devices you may need to change permissions on the cfg file as well as the data file in /usr/local/www/mgmt/mrtg/data, including removal of the rrd file if necessary.

Errors

"Lock wait timeout exceeded"

delete error - Can't delete a2206e24: DBD::mysql::st execute failed: Lock wait timeout exceeded; try restarting transaction [for Statement "DELETE FROM invoice WHERE inv_ref=? "] at /usr/local/lib/perl5/site_perl/5.6.1/DBIx/ContextualFetch.pm line 51. at /usr/local/www/mgmt/Plugin/Billing.pm line 1934

This is the result of an unclean submit/commit. Usually from an error or a double click on something that should have been single click. To clear this up, restart the database:

mail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
mysqldmail /usr/local/www/scripts#

It takes a minute to shutdown. I keep running the command until it says it isn't running, then I start it:

mail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
 mysqldmail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
 mysqldmail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
mysql-server isn't running
mail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh start
 mysqldmail /usr/local/www/scripts#

Account Manager (AM)

Setup / organization / notes

• domain: https://secure.johncompanies.com/am
• webroot: /usr/local/www/am

Directory structure:

• am: contains the main handler AM.pm, the authentication (AMAuth.pm), and password reset (AMPassR.pm) handlers
• am/Lib: contains the log module AMLog.pm
• am/Log: contains the log am.log, and ats activity log ats.log
• am/Plugin: contains the plugin module AMP.pm and the form fill in module FillInForm.pm
• am/html: contains all web pages
• am/static: contains static, non-interpreted content like graphics and javascripts

The Account Manager was designed and intended to allow customers to do the following:

• see their services and various details about it
  • IP
  • type service
  • cost of service
  • link to control panel (virtuozzo CT only)
• to manage their service level
• to power cycle their server (if at i2b)
• to see their bandwidth usage and allocation
• to manage their contact info
• to review their billing history
• to update their billing method info

It is accessed via our secure website. Customers must login using their CID (colNNNNN) and a password which is created specifically for use with the AM. It is not tied in any way to their server, though some customers may be confused by that or think that changing the password in one location will affect the other.

JC staff may login to any customer's AM by using a special (all access) password.

In order to access the AM for any given customer, a few things must be in place:

1. they must have a password issued for their AM access. In order to determine if a password has been set you can look at the "AM pass" (jc:customers.am_auth) field in the customer's page in mgmt. If it says "(hidden)" a password has been established. If instead you see 3 links:
   reset activate
   activate+email
   that means the customer has no password and access has not been enabled.
2. the "owner email" (jc:customers.owner_email) field must contain the owner's email address. If a customer has been issued the pass to their AM but an owner email has not been selected, they will be prompted to select one from the existing contacts upon logging in for the first time. The selected owner will be emailed. It's for this reason that if you access the AM for a customer with the global pass, you should make sure a owner has been selected before hand (enter the info in mgmt) and don't select it in the AM, lest you want them to get a random email.

If no password has been established (or it has been forgotten- it's stored as a 1 way hash) it must be reset. The customer may do this themselves via the password reset page (linked off the AM login page). This will require them to have on file and use/match the owner_email field. You can also reset the password via mgmt via 2 methods:

1. reset: will just reset their password and show it to you in mgmt. It will also give you handy instructions (commands) on how to place their CID and AM password in a file on their VPS. We prefer to do it this way since we then don't have to decide if the person requesting access is authorized or not. If we place the CID/password in a file on their VPS and make it viewable by root only, then only someone with root access could get to that and if you have root access, you're considered authorized.
2. activate+email: this will set/reset their password and send an email to the customer with access instructions and it will instruct them to find the AM access info in the /root/ampass on their VPS- which you will create for them as soon as you click this link. If the customer has only colo's, the site will recognize that and email them the instructions/password.
3. activate: this will enable AM access. The customer may then use the password reset function to get a password set for access.

There is lots of contextual/hover help throughout the AM to help customers understand what they're looking at.

Note - if a(n old) customer with no AM access gets a new VPS, the welcome email will indicate they have AM access established- this isn't true. You will need to at least activate the AM for them and possibly also reset their pass and update the ampass file accordingly.

Usage

Dashboard

This is the main/landing page for the AM. It has several sections:

• Announcements: This is where we make available any announcement we might have for the customer. If the customer had a billing issue, it would be listed here.

• Services: lists the services (servers) which the customer has and basic info about it (service type, IP, hostname, OS)

• Bandwidth: shows condensed usage/allocation, overage and how we deal with overage (per the customer's preference).

• Contacts: shows the contacts on file for the account and their role.

Services

This page shows in detail what services they have with us. For instance, their package and what it comes with in terms of IP, disk space, bandwidth, price. There is a link for them to change service on this page- this just trigger's an email to support so they can't actually change their service instantly.

If they have a colo @ i2b, the power controls are available on this page. The status of the power outlet to which their server is connected is displayed ("Current Status") and if the ATS is responding (see below) 2 radio options are given for controlling the port: "Off" and "On". Pretty self explanatory, "Off" turns the port off and "On" turns it on. In order to power cycle the server, the will need to first select the "Off" option and click the "Submit" button. A window will popup and display the status of the port until the status switches to off (it will auto refresh every 10sec). To power back on, they just need to set the option to "On" and click "Submit".

If the customer has a virtuozzo VPS, the link to their VZ control panel is provided.

Bandwidth

This page allows the customer to pull up bandwidth usage statistics. The reporting of those statistics is available in several flavors and can be customized to show data based on:

• date range
• server
• IP(s)
• data flowing in a particular direction
• protocol (ICMP, TCP, UDP)
• port(s)

Further, data can be output as:

• daily bar graph
• 15min granularity bar graph
• pie chart
• table/raw

The only thing to keep in mind here is that anything but the daily bar graph query will take up a bit more time as all other queries come from much larger tables and thus require more time to seek through the database.

The graphs generated by this activity actually generate files which live on the server for a period of time, and are removed by cronjob.

Billing

This screen displays the payment method currently active for the customer, and allows them to update that info (in the case of credit card payments).

It also lists their payment history, oldest to newest, and their outstanding balance.

It displays their ongoing JC charges.

If they have a paypal subscription setup it displays that.

Finally, if they have an outstanding (paypal) invoice, it will display here and give them a link to (pay) it.

Account Info

Allows the customer to see / update:

• account owner info
• contact info
• the account manager password
• bandwidth preferences

Any modifications to these settings will require the user to request a security token to be emailed to the owner (jc:customers.owner_email) which they are required to provide in order to make updates to this page. Once the customer provides this key and goes on to edit the info it is no longer valid and any subsequent visit to the edit page will require the generation of a new security token. Obviously, this requires the customer to have the ability to retrieve email at the owner's email address on file.

Support

This page allows the customer to quickly send us a message. They are allowed to send it as/from one of the contacts on file (or provide a new one) and they can indicate which server the issue is regarding. The email is cc'd back to the customer for their records. This form is intelligent and will send the email to the right place based on the server/product they select- support@ or linux@

Problems

Power management: Status and control temporarily unavailable

ATS isn't responding. See Rebooting and Recovering

Power control doesn't work

If after requesting a change in power status, the popup status window never reflects a change to the requested status, that means the ATS isn't responding and the customer should try setting the power again. If that still doesn't work, the ATS is out and needs to be reset. See Rebooting and Recovering

Database

The database behind AM/Signup/MGMT runs on mysql on the mail server.