Management System / Public Website / Signup / Account Manager

Overview

These systems are all running under the same apache instance on the mail server.

The main ingredients are: apache 1.3.31, mysql, perl, mod_perl and template toolkit.

Apache is listening on 69.55.230.9 ports 80 and 443

The webserver can be stopped with:

apachectl stop

and must be started with:

apachectl startssl

If you run apachectl start none of the ssl-enabled pages will work.

The database may be stopped and started as follows:

/usr/local/etc/rc.d/mysql-server.sh stop
/usr/local/etc/rc.d/mysql-server.sh start

• webroot: /usr/local/www/
• logs: /var/log/httpd-error.log /var/log/httpd-access.log
• Apache config:

/usr/local/etc/apache/httpd.conf:

-SNIP-

<Directory /usr/local/www>
 Options none
</Directory>


<VirtualHost *>
  SSLDisable
  DocumentRoot /usr/local/www/jc_pub
  ServerName johncompanies.com
  ServerAlias www.johncompanies.com
  ServerAlias newwww.johncompanies.com

  <Directory /usr/local/www/jc_pub>
    Options FollowSymLinks
  </Directory>

  Redirect /collocation http://www.johncompanies.com
  Redirect /colocation http://www.johncompanies.com

  ScriptAlias /cgi-bin/ "/usr/local/www/jc_pub/cgi-bin/"
  <Directory /usr/local/www/jc_pub/cgi-bin>
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
    SetHandler None
  </Directory>

</VirtualHost>

<VirtualHost *>
  ServerName mini.johncompanies.com
  SSLDisable
  DocumentRoot /usr/local/www/mini

  <Directory /usr/local/www/mini>
    Options FollowSymLinks
  </Directory>

</VirtualHost>


<VirtualHost _default_:443>

  DocumentRoot "/usr/local/www"

  DocumentRoot /usr/local/www
  <Directory /usr/local/www>
    Deny from All
  </Directory>
  <Directory /usr/local/www/mgmt>
    Allow from All
  </Directory>
  <Directory /usr/local/www/am>
    Allow from All
  </Directory>
  <Directory /usr/local/www/signup>
    Allow from All
  </Directory>
  <Directory /usr/local/www/images>
    Allow from All
  </Directory>
  <Directory /usr/local/www/media>
    Allow from All
  </Directory>
  <Files favicon.ico>
    Allow from All
  </Files>

  SSLEngine on
  SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL

#  SSLCertificateFile         /usr/local/etc/apache/mail.cert
#  SSLCertificateKeyFile     /usr/local/etc/apache/mail.key
  #SSLCertificateFile         /usr/local/etc/apache/ssl.crt/secure.crt
  SSLCertificateFile         /usr/local/etc/apache/ssl.crt/secure.johncompanies.com.crt
  SSLCertificateChainFile    /usr/local/etc/apache/ssl.crt/gd_bundle.crt
  SSLCertificateKeyFile     /usr/local/etc/apache/ssl.key/secure.key

  PerlModule Apache

  # debug stuff
  PerlWarn On
  PerlTaintCheck Off
  PerlModule Apache::StatINC
  PerlInitHandler Apache::Reload
# CustomLog /usr/local/apache/logs/access_log.mgmt common
# ErrorLog /usr/local/apache/logs/error_log.mgmt

  PerlModule Apache::DBI
#  PerlModule Mgmt
#  PerlModule Auth

  PerlSetEnv MGMT_BASE /usr/local/www/mgmt
  PerlSetEnv COMMON_BASE  /usr/local/www/common
  PerlSetEnv SIGNUP_BASE  /usr/local/www/signup
  PerlSetEnv AM_BASE /usr/local/www/am
  PerlSetEnv MGMT_INDEX /mgmt/index.html
  PerlSetEnv SIGNUP_INDEX /signup/step1.html
  PerlSetEnv AM_INDEX /am/dashboard.html
  PerlSetEnv MGMT_LOG_LEVEL debug
  PerlSetEnv MGMT_LOG_FILE /usr/local/www/mgmt/Log/mgmt.log
  PerlSetEnv SIGNUP_LOG_LEVEL info
  PerlSetEnv SIGNUP_LOG_FILE /usr/local/www/signup/Log/signup.log
  PerlSetEnv AM_LOG_LEVEL debug
  PerlSetEnv AM_LOG_FILE /usr/local/www/am/Log/am.log
  PerlSetEnv PP_AUTH_TOKEN Pe3aLk5GdMblAyyLAv5vNYqipcynWdZJKdw1CmcGcIdOz74ujMrDYIov27i
  PerlSetEnv PP_URL 'https://www.paypal.com/cgi-bin/webscr'
  PerlSetEnv PP_EMAIL 'payments@johncompanies.com'
  PerlSetEnv JC_DOMAIN 'secure.johncompanies.com'
  PerlSetEnv CC_LOG_FILE /usr/local/www/mgmt/Log/cc.log
  PerlSetEnv DEV 0
  PerlRequire /usr/local/www/common/conf/startup.pl

  <Directory "/usr/local/www/mgmt">
    SetHandler perl-script
    PerlHandler Mgmt
    PerlSetVar JCMGMTPath /mgmt
    PerlSetVar JCMGMTLoginScript /mgmt/login.html

    <Files LOGIN>
      AuthType Auth
      AuthName JCMGMT
      SetHandler perl-script
      PerlHandler Auth->login
    </Files>

    <FilesMatch ".*\.html$|.*\.cgi|.*\.png">
      AuthType Auth
      AuthName JCMGMT
      PerlAuthenHandler Auth->authenticate
      PerlAuthzHandler  Auth->authorize
      require valid-user
    </FilesMatch>
  </Directory>

  <Directory "/usr/local/www/am">
    SetHandler perl-script
    PerlHandler AM
    PerlSetVar JCAMPath /am
    PerlSetVar JCAMLoginScript /am/login.html
    #PerlSetVar JCAMExpires +1m

    <Files LOGINAM>
     AuthType AMAuth
     AuthName JCAM
     SetHandler perl-script
     PerlHandler AMAuth->login
    </Files>

    <Files PASSRESET>
     SetHandler perl-script
     PerlHandler AMPassR
    </Files>

    <FilesMatch ".*\.html$|.*\.cgi">
     AuthType AMAuth
     AuthName JCAM
     PerlAuthenHandler AMAuth->authenticate
     PerlAuthzHandler AMAuth->authorize
     require valid-user
    </FilesMatch>
  </Directory>

  <Directory /usr/local/www/mgmt/static>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/am/static>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/mgmt/bwgraphs>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/am/bwgraphs>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/images>
    SetHandler None
  </Directory>

  <Directory /usr/local/www/media>
    SetHandler None
  </Directory>
  <Directory /usr/local/www>
    Options FollowSymlinks
  </Directory>

  ScriptAlias /mgmt/cgi/ "/usr/local/www/mgmt/cgi/"

  <Directory /usr/local/www/mgmt/cgi>
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
    SetHandler None
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
  </Directory>

  <Directory "/usr/local/www/signup">
    SetHandler perl-script
    PerlHandler Signup
  </Directory>

  Alias /mgmt/mrtg "/usr/local/www/mgmt/mrtg/data"
  <Directory /usr/local/www/mgmt/mrtg/data/>
    DirectoryIndex index.cgi
    SetHandler None
    Options ExecCGI
    AddHandler cgi-script .cgi
  </Directory>

  Alias /mgmt/rrd "/usr/local/www/mgmt/mrtg/rrd"
  <Directory /usr/local/www/mgmt/mrtg/rrd/>
    DirectoryIndex index.html
    SetHandler None
  </Directory>

  ScriptAlias /mgmt/bb/cgi-bin/ /usr/home/bb/bbsrc/bb1.9i-btf/web/
  Alias /mgmt/bb "/usr/home/bb/bbsrc/bb1.9i-btf/www"
  <Directory /usr/home/bb/bbsrc/bb1.9i-btf/www/gifs>
    SetHandler None
  </Directory>
  <Directory /usr/home/bb/bbsrc/bb1.9i-btf/web>
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
    PerlSetVar JCMGMTLoginScript /mgmt/login.html
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
  </Directory>
  <Directory /usr/home/bb/bbsrc/bb1.9i-btf/www>
    PerlSetVar JCMGMTLoginScript /mgmt/login.html
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
  </Directory>

  Alias /mgmt/awstatsclasses "/usr/local/www/mgmt/awstats/wwwroot/classes/"
  Alias /mgmt/awstatscss "/usr/local/www/mgmt/awstats/wwwroot/css/"
  Alias /mgmt/awstatsicons "/usr/local/www/mgmt/awstats/wwwroot/icon/"
  ScriptAlias /mgmt/awstats/ "/usr/local/www/mgmt/awstats/wwwroot/cgi-bin/"
  Alias /mgmt/icon/ "/usr/local/www/mgmt/awstats/wwwroot/icon/"

  <Directory "/usr/local/www/mgmt/awstats/wwwroot/icon">
    SetHandler None
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>


  <Directory "/usr/local/www/mgmt/awstats/wwwroot">
    PerlSetVar JCMGMTLoginScript /mgmt/login.html
    AuthType Auth
    AuthName JCMGMT
    PerlAuthenHandler Auth->authenticate
    PerlAuthzHandler  Auth->authorize
    require valid-user
    SetHandler None
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Template Toolkit

http://www.template-toolkit.org/docs/manual/index.html

All our dynamic content pages (i.e. anything but jcpub) makes use of the Template Toolkit (TT) framework to create/display our pages. This package allows us to create customied apache handlers via the use of mod_perl. The result is an easy to deploy, easy to develop, flexible and efficient platform.

All these sites are organized under /usr/local/www and consist/rely on several components:

• common/conf/startup.pl - this contains all the use/require code that brings in all the required libraries and modules required to run our sites and TT. It is run once as apache starts up and applies to all sites.

• common/conf/Lib - this directory contains all common libraries, primarly those dealing with database access and access to individual tables.

• mysql:jc - the main table containing everything for mgmt/am/signup sites is contained in the jc database. All data is accessed via the mysql user jc - this is so we can impose restrictions on what that user may do. That access info is stored in common/Lib/DBI.pm

The exception to that is the traffic database which used to be stored on mail, but has since been exported to run locally on the bwdb server itself. i.e. mgmt/am contact the remote traffic database running elsewhere for that data. That access data is stored in common/Lib/DBI_BWDB.pm

Part of TT's efficiency comes from static and natively compiled code/pages. Those pages are stored in:

• mgmt: /tmp/mgmt_templates/
• am: /tmp/am_templates
• signup: /tmp/signup_templates

You could safely delete all of these and apache will just recreate them - though the directory holding the templates must exist, with the right permissions.

To make editing of pages easier we enable apache modules/directives which direct apache to rebuild these pages when it notices there's a difference (httpd.conf):

 PerlModule Apache::StatINC
 PerlInitHandler Apache::Reload

However, any changes made to anything under Lib requires an immediate apache restart to take effect and to keep the site running- any changes w/o a restart will break the site.

Among the directives in httpd.conf you will see a series of variables which are important as they, among other things, direct the mod_perl handlers where certain data is stored:

This is where all the files for a particular set of pages and unique handler are located:

 PerlSetEnv MGMT_BASE /usr/local/www/mgmt
 PerlSetEnv COMMON_BASE  /usr/local/www/common
 PerlSetEnv SIGNUP_BASE  /usr/local/www/signup
 PerlSetEnv AM_BASE /usr/local/www/am

This is the default page to present, if none is specified:

 PerlSetEnv MGMT_INDEX /mgmt/index.html
 PerlSetEnv SIGNUP_INDEX /signup/step1.html
 PerlSetEnv AM_INDEX /am/dashboard.html

Indicates the verbosity and location of the handler logfile (these log files only contain logging which we do internally from our code- these differ from the apache logs which log simple hits and server errors):

 PerlSetEnv MGMT_LOG_LEVEL debug
 PerlSetEnv MGMT_LOG_FILE /usr/local/www/mgmt/Log/mgmt.log
 PerlSetEnv SIGNUP_LOG_LEVEL info
 PerlSetEnv SIGNUP_LOG_FILE /usr/local/www/signup/Log/signup.log
 PerlSetEnv AM_LOG_LEVEL debug
 PerlSetEnv AM_LOG_FILE /usr/local/www/am/Log/am.log

Variables used for pp API access:

 PerlSetEnv PP_AUTH_TOKEN Pe3aLk5GdMblAyyLAv5vNYqipcynWdZJKdw1CmcGcIdOz74ujMrDYIov27i
 PerlSetEnv PP_URL 'https://www.paypal.com/cgi-bin/webscr'
 PerlSetEnv PP_EMAIL 'payments@johncompanies.com'

Our base url:

 PerlSetEnv JC_DOMAIN 'secure.johncompanies.com'

Our credit card info logging file:

 PerlSetEnv CC_LOG_FILE /usr/local/www/mgmt/Log/cc.log

Indicates we are/not in a dev environment:

 PerlSetEnv DEV 0

Public Website (jcpub)

• domain: http://www.johncompanies.com http://johncompanies.com
• webroot: /usr/local/www/jc_pub

Our public-facing website is all static, standard HTML. We have some light javascripting on some pages, but by in large it's a very WYSIWYG site setup.

Signup (signup)

Setup / organization / notes

• domain: https://secure.johncompanies.com/signup/step1.html
• webroot: /usr/local/www/signup

Directory structure:

• am: contains the main handler Signup.pm and the ipn activity log ipn.log
• signup/Lib: contains the log module SLog.pm
• signup/Log: contains the log signup.log, and cardit card activity log cc.log
• signup/Plugin: contains the plugin module AMP.pm and the form fill in module FillInForm.pm
• signup/html: contains all web pages

Management System (mgmt)

Setup / organization / notes

• domain: https://secure.johncompanies.com/mgmt
• webroot: /usr/local/www/mgmt

Directory structure:

• mgmt: contains the main handler Mgmt.pm and the authentication (Auth.pm) handler, the bigbrother status file bb.html, the credit card gpg key pubring.gpg
• mgmt/Lib: contains the log module MLog.pm
• mgmt/Log: contains the log am.log, and ats activity log ats.log
• mgmt/Plugin: contains the plugins required for the various mgmt pages
• mgmt/html: contains all web pages
• mgmt/static: contains static, non-interpreted content like graphics and javascripts
• mgmt/awstats: our stats package. Not controlled by TT, just lives here so you have to authenticate in mgmt to see it
• mgmt/bwgraphs: deprecated? b/w graphing libraries and code
• mgmt/mrtg: mrtg graphs. Not controlled by TT, just lives here so you have to authenticate in mgmt to see it

• users table: jc.users

mrtg

• domain: https://secure.johncompanies.com/mgmt/mrtg/index.cgi/ https://secure.johncompanies.com/mgmt/mrtg/switch.cgi?s=switch-p3&path=
• webroot: /usr/local/www/mgmt/mrtg

All configuration is done via *.cfg files. The main load graph is found in mrtg1.cfg All other config files are for various switches. Switch config files are rebuilt out of a cron jobs running on mail. This ensures if we change a port name (desc) that the mrtg we look at has the latest info. So if you want to change port naming, please do it in the switch itself. If you have problems getting new devices setup or change existing devices you may need to change permissions on the cfg file as well as the data file in /usr/local/www/mgmt/mrtg/data, including removal of the rrd file if necessary.

Errors

"Lock wait timeout exceeded"

delete error - Can't delete a2206e24: DBD::mysql::st execute failed: Lock wait timeout exceeded; try restarting transaction [for Statement "DELETE FROM invoice WHERE inv_ref=? "] at /usr/local/lib/perl5/site_perl/5.6.1/DBIx/ContextualFetch.pm line 51. at /usr/local/www/mgmt/Plugin/Billing.pm line 1934

This is the result of an unclean submit/commit. Usually from an error or a double click on something that should have been single click. To clear this up, restart the database:

mail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
mysqldmail /usr/local/www/scripts#

It takes a minute to shutdown. I keep running the command until it says it isn't running, then I start it:

mail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
 mysqldmail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
 mysqldmail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh stop
mysql-server isn't running
mail /usr/local/www/scripts# /usr/local/etc/rc.d/mysql-server.sh start
 mysqldmail /usr/local/www/scripts#

Account Manager (AM)

Setup / organization / notes

• domain: https://secure.johncompanies.com/am
• webroot: /usr/local/www/am

Directory structure:

• am: contains the main handler AM.pm, the authentication (AMAuth.pm), and password reset (AMPassR.pm) handlers
• am/Lib: contains the log module AMLog.pm
• am/Log: contains the log am.log, and ats activity log ats.log
• am/Plugin: contains the plugin module AMP.pm and the form fill in module FillInForm.pm
• am/html: contains all web pages
• am/static: contains static, non-interpreted content like graphics and javascripts

The Account Manager was designed and intended to allow customers to do the following:

• see their services and various details about it
  • IP
  • type service
  • cost of service
  • link to control panel (virtuozzo CT only)
• to manage their service level
• to power cycle their server (if at i2b)
• to see their bandwidth usage and allocation
• to manage their contact info
• to review their billing history
• to update their billing method info

It is accessed via our secure website. Customers must login using their CID (colNNNNN) and a password which is created specifically for use with the AM. It is not tied in any way to their server, though some customers may be confused by that or think that changing the password in one location will affect the other.

JC staff may login to any customer's AM by using a special (all access) password.

In order to access the AM for any given customer, a few things must be in place:

1. they must have a password issued for their AM access. In order to determine if a password has been set you can look at the "AM pass" (jc:customers.am_auth) field in the customer's page in mgmt. If it says "(hidden)" a password has been established.
2. the "owner email" (jc:customers.owner_email) field must contain the owner's email address. If a customer has been issued the pass to their AM but an owner email has not been selected, they will be prompted to select one from the existing contacts upon logging in for the first time. The selected owner will be emailed. It's for this reason that if you access the AM for a customer with the global pass, you should make sure a owner has been selected before hand (enter the info in mgmt) and don't select it in the AM, lest you want them to get a random email.

If no password has been established (or it has been forgotten- it's stored as a 1 way hash) it must be reset. The customer may do this themselves via the password reset page (linked off the AM login page). This will require them to have on file and use/match the owner_email field. You can also reset the password via mgmt via 2 methods:

1. reset: will just reset their password and show it to you in mgmt. It will also give you handy instructions (commands) on how to place their CID and AM password in a file on their VPS. We prefer to do it this way since we then don't have to decide if the person requesting access is authorized or not. If we place the CID/password in a file on their VPS and make it viewable by root only, then only someone with root access could get to that and if you have root access, you're considered authorized.
2. activate+email: this will set/reset their password and send an email to the customer with access instructions and it will instruct them to find the AM access info in the /root/ampass on their VPS- which you will create for them as soon as you click this link. If the customer has only colo's, the site will recognize that and email them the instructions/password.

There is lots of contextual/hover help throughout the AM to help customers understand what they're looking at.

Usage

Dashboard

This is the main/landing page for the AM. It has several sections:

• Announcements: This is where we make available any announcement we might have for the customer. If the customer had a billing issue, it would be listed here.

• Services: lists the services (servers) which the customer has and basic info about it (service type, IP, hostname, OS)

• Bandwidth: shows condensed usage/allocation, overage and how we deal with overage (per the customer's preference).

• Contacts: shows the contacts on file for the account and their role.

Services

This page shows in detail what services they have with us. For instance, their package and what it comes with in terms of IP, disk space, bandwidth, price. There is a link for them to change service on this page- this just trigger's an email to support so they can't actually change their service instantly.

If they have a colo @ i2b, the power controls are available on this page. The status of the power outlet to which their server is connected is displayed ("Current Status") and if the ATS is responding (see below) 2 radio options are given for controlling the port: "Off" and "On". Pretty self explanatory, "Off" turns the port off and "On" turns it on. In order to power cycle the server, the will need to first select the "Off" option and click the "Submit" button. A window will popup and display the status of the port until the status switches to off (it will auto refresh every 10sec). To power back on, they just need to set the option to "On" and click "Submit".

If the customer has a virtuozzo VPS, the link to their VZ control panel is provided.

Problems

Power management: Status and control temporarily unavailable

ATS isn't responding. See ATS#Rebooting_and_Recovering