NetHere
NetHere[edit]
URL: http://toolbox.nethere.com
Access to servers is restricted to admin-1.nethere.net. Access to admin-1.nethere.net is restricted to our office and nat.johncompanies.com.
JCI Hosting acquired the web-hosting customers of NetHere. (Previously, NetHere had aquired customers from Simply Internet, and Z-net).
Customer billing/info are being imported from NetHere to JCI database
URL: https://secure.johncompanies.com/mgmt/index.html
Nethere has several tools for management. These are restricted to only allow
access from the office, or a few admin's homes.
cacti [1] nagios[2] toolbox [3] helpdesk [4] support FAQ [5] knowledge base [6] phpmyadmin [7]
Virtual Hosts[edit]
66.63.129.2 - New HP DL360 fpweb-2.nethere.net unixweb-6.nethere.net mta-db.mail.nethere.net scan-1.mail.nethere.net 66.63.129.4- vmware9.eng mysqldb-1.webhost.nethere.net 66.63.129.91 lists-1.nethere.net sb-2.nethere.net mx-1.nethere.net- spooling server mailx-1.nethere.net- web server outbound mail server 66.63.129.101 (c.host.nethere.net) ahi.nethere.net- ldap mailbox-4 mta-2.mail.nethere.net relay-2.mail.nethere.net unixweb-8.nethere.net 66.63.129.102 (vm2.eng.nethere.net) koi.nethere.net- off- realserver prov-1.nethere.net- cp.nethere.net, cp.znet.net, cp.simplyweb.net scribe.nethere.net- prov.nethere.net, scribe.nethere.net tetra.nethere.net- nethere.com, znet.com, simplyweb.com web-2.inboxhq.net- helpdesk webmail-1.nethere.net 66.63.129.103 andromeda.nethere.net (ns2.nethere.net) nscache-2.nethere.net nsrbl-2.nethere.net mailbox-2.nethere.net scan-2.mail.nethere.net 66.63.129.104 eel.nethere.net- ldap mta-1.mail.nethere.net nscache-1.nethere.net nsrbl-1.nethere.net phoenix.nethere.net (ns1.nethere.net) relay-1.mail.nethere.net unixweb-2.nethere.net 66.63.129.105 home-1.nethere.net ntweb-11.nethere.net ntweb-6.nethere.net cart32.nethere.net (web-3.schedulecafe.com) 66.63.129.106 6499MB/16381 18.32GB/268.25 free admin-1.nethere.net admin-2.nethere.net pike.nethere.net- cacti, nagios- off shark.nethere.net- ldap ntdb-1.nethere.net winrestore64 207.167.93.106- vmware6.eng 4156MB/8185MB 45.94GB/267GB free ntweb-2.nethere.net unixweb-12.nethere.net (no web sites on server) mailbox-1.nethere.net 207.167.93.108- vmware8.eng mailx-2.nethere.net mx-2.nethere.net 207.167.93.110- vmware10.eng unixweb-11.nethere.net
Hardware Hosts[edit]
Cab 6-08 siron-3 (off) siron-2 siron-1 106 104 sndg-br-1 sndg-br-2 mailbox-2.nethere.net (off - virtualized) mailbox-3.nethere.net mailbox-1.nethere.net (off - virtualized) scan-1 (OFF) scan-2 129.2 102 101 105 103 Backup-2 Backup-1
Cab 5-02 ntweb-3 unixweb-7 unixweb-3 unixweb-2 (now virtualized) unixweb-10 ntdb-2 or 3? vmware6.eng
Admin Access[edit]
Access to the Admin Systems are protected by firewalls and application filters based on IP addresses.
Main Firewall[edit]
The Main firewall is on sndg-cr-1. The main firewall is a Cisco 3750.
IP Access List Editing[edit]
1. enable
2. show ip access-lists access-list-name
show ip access-list BLOCKED
3. configure terminal
4. ip access-list resequence access-list-name starting-sequence-number increment
5. ip access-list {standard | extended} access-list-name
ip access-list extended BLOCKED
6. sequence-number permit source source-wildcard
sequence-number deny protocol source source-wildcard destination destination-wildcard [precedence precedence] [tos tos] [log] [time-range time-range-name] [fragments]
no sequence-number
7. end
8. show ip access-lists access-list-name
Host Firewall[edit]
On the servers, there is often a firewall such as ipfw, ipf, or pf.
Toolbox[edit]
From root@admin-1 connect to scribe
ssh scribe
Stop the scrolling of messages to the screen
/etc/rc.d/syslogd stop
Add the new IP address to the pf config.
vi /etc/pf.conf
Reload the firewall rules
pfctl -F all -f /etc/pf.conf
Add the new IP address to the apache config.
cd /nethere/conf/apache/conf/vhosts co -l prov.nethere.net scribe.nethere.net.common vi prov.nethere.net vi scribe.nethere.net.common ci -u prov.nethere.net scribe.nethere.net.common
Reload the apache config
apachectl restart
Restart the syslog daemon
/etc/rc.d/syslogd start
Nagios[edit]
From root@admin-1 connect to pike
ssh nagios
Add the new IP address to the apache config.
cd /nethere/conf/apache/conf/vhosts co -l nagios.nethere.net cacti.nethere.net mrtg.nethere.net vi nagios.nethere.net cacti.nethere.net mrtg.nethere.net ci -u nagios.nethere.net cact.nethere.net mrtg.nethere.net
Reload the apache config
apachectl restart
Application Filter[edit]
In the applications, there is often a restriction in the host application configuration file.
vi /usr/local/etc/apache/conf/vhosts/<host>.conf
Restart the application
apachect restart
NetHere/zNET/simplyweb Cert replacement[edit]
If you wish to use the old CSR, Global Sign keeps the old CSR, so all you have to replace is the Cert.
openssl req -nodes -newkey rsa:2048 -keyout wild.nethere.key -out wild.nethere.csr openssl req -nodes -newkey rsa:2048 -keyout wild.simplyweb.key -out wild.simplyweb.csr openssl req -nodes -newkey rsa:2048 -keyout wild.znet.key -out wild.znet.csr
https://www.globalsign.com/en/
webmail-1.mail (webmail for nh, si, and zn) /nethere/conf/apache/pki/ssl.crt
prov-1.mail (cp for nh, si, and zn) /nethere/conf/apache/pki/ssl.crt
mta-1 and mta-2 (smtp) admin-2:/dist/files/nhmta/nethere/conf/postfix/nh/ssl
/nethere/conf/postfix/nh/ssl
/nethere/conf/postfix/si/ssl
/nethere/conf/postfix/zn/ssl
mailbox-1, mailbox-2, and mailbox-3 (pop3 and imap) (courier-imap)
/nethere/conf/courier-imap/nh
/nethere/conf/courier-imap/si
/nethere/conf/courier-imap/zn
(smtp)
Provisioning[edit]
Webservers[edit]
OS: FreeBSD 4.11-RELEASE-p10 #23
General[edit]
unixweb-1 through 8 All have the same configuration except for the following exceptions: unixweb-4: ZN Front Page server (depreciated) unixweb-5: ZN Front Page, Miva Merchant server unixweb-6: SI shared counter server (for SI sites): /www/lucy.inetworld.net/htdocs/cgi-bin/Count.cgi Server Software Installed Apache # httpd -v Server version: Apache/2.0.55 Server built: Apr 5 2006 17:04:01 PHP # /usr/local/bin/php4 -v PHP 4.4.2 (cgi-fcgi) (built: Mar 2 2006 09:31:57) Miva Merchant (unixweb-5) MySQL # mysqladmin version -u root -p Server version 4.0.25 Apache configurations are in the following directories: /nethere/conf/apache/conf/nvhosts (name based hosting); naming convention is www.domain.tld /nethere/conf/apache/conf/vhosts (ip based,SSL); naming convention is www.domain.tld (IP information) and www.domain.tld.common (general site information) NetHere specific scripts are located in: /nethere/sbin Checking Server/site status via mod_status apache module: # apachectl stop # /usr/local/etc/rc.d/httpd.sh start-status URL: http://unixweb-#/status (replace # with the unixeweb number) ** NOTE: After viewing status, do the following, otherwise other users can also view status by using .htaccess. # apachectl stop # apachectl startssl
Provisioning[edit]
Provisioning new sites:
- All website provisioning is done via NH script (webadd) on sawfish (admin-1.nethere.net) ***
- /nethere/sbin/webadd -h for usage
Note: We've discontinued new website provisioning on all servers except for unixweb-6, unless otherwise noted for domain
FTP Password Adjustment[edit]
Unix Server[edit]
To change an FTP password:
On the customers server as root do
passwd <username>
(The password prompt will not echo entry)
Windows Server[edit]
To change an FTP password:
Find the users login (ie aa5014) in the provisioning tool. Use Remote Desktop to get into the server. On Desktop find FTP application Icon. Use that tool to update the user's password.
Domain Aliasing[edit]
For domain aliasing:
1) Edit the /named/named.master on phoenix (ns1.nethere.net)
a) Checkout the file; open with vi
# co -l named.master
# vi named.master
b) find the domain that's serving as the master domain, then, following the general format of the file to add the domain aliases under the master domain entry, using the master domain zone file
i.e.
--cut--
zone "domain.tld" { type master; file "master/domain.tld"; };
zone "aliased_domain.tld" { type master; file "master/master_domain.tld"; };
--cut--
c) save the file, then check-in:
# ci -u named.master < /dev/null
d) update the zone records, using the makefile in /named; check for errors
# make new-zone
# tail /var/log/named
**Note: some domains are IP based virtually hosted (i.e. SSL certificate). This requires a modification to the standard aliasing procedure. If this is the case, do the following
e) Create a new zone file called "domain.tld-alias" in /named/master, copy current domain.tld zone file to the domain.tld-alias file, and adjust the "website" records, removing whatever IP based information is there, and replacing with the appropriate $INCLUDE name-based host template.
e.g.
tropicalshade.net:
--cut--
;; ntweb-4
@ IN A 66.63.136.4
www IN A 66.63.136.4
--cut--
tropicalshade.net-alias
--cut--
;; website
$INCLUDE master/nvhost.ntweb-4
--cut--
f) Adjust the named.master zone file in /named accordingly:
i.e.
--cut--
zone "aliased_domain.tld" { type master; file "master/master_domain.tld-alias"; };
--cut--
2) Edit the Apache configuration for the domain on the web server the master domain is hosted on
a) Script is /nethere/sbin/webalias
# /nethere/sbin/webalias -h (for usage)
e.g. # /nethere/sbin/webalias -d www.originaldomain.tld -a www.domainalias.tld
CGI Scripting[edit]
For security, we've implemented suexec on all sites. All CGI *must* be placed in the cgi-bin/ (and/or cgibin/ directory for unixweb-4, 5) directory
CGI permissions break down as follows:
Ownership: user=username, group=webuser Permission: cgi-bin/ = 755; files = 755
suexec errors can be found here: /www/default/logs/suexec_log
cgi errors for a domain can be found in the main apache error logs for the domain: /www/www.domain.tld/logs/error_log
More information on CGI scripting can be found here: http://httpd.apache.org/docs/1.3/howto/cgi.html
Formmail Provisioning[edit]
Formmail is provisioned via NH script (newformmail) on server domain is hosted on.
- /nethere/sbin/newformmail -h for usage
Usage: newformmail [-h] [-d domain] [-i ip_address] [-e "email1 email2"]
Disabling a site[edit]
To disable/enable a website, there is a script (webstatus) located on each webserver: /nethere/sbin/webstatus -h for usage
1) Disable:
# /nethere/sbin/webstatus -r -d -s www.domain.tld
2) Enable:
# /nethere/sbin/webstatus -r -e -s www.domain.tld
Stats Provisioning[edit]
We utilize webalizer for statistics.
Stats are provisioned via NH scripts on server domain is hosted on.
1) Provision stats for the domain (webaddstats_unix):
- /nethere/sbin/webaddstats_unix -d www.domain.tld
2) Run stats for the domain (runstats_unix):
- /nethere/sbin/runstats_unix www.domain.tld
More information on Webalizer can be found here: http://www.mrunix.net/webalizer README: ftp://ftp.mrunix.net/pub/webalizer/README
Removing a site[edit]
1) Site removal is primarily done via a NH script (webdel) on the webserver the domain is hosted on:
# /nethere/sbin/webdel -h for usage # /nethere/sbin/webdel -r -d www.tldomain.com
2) Update DNS accordingly (ns1.nethere.net)
a) Checkout /named/named.master
# co -l named.master
b) Remove line containing domain
c) Checkin /named/named.master
# ci -u named.master < /dev/null
d) Update zones
# make new-zone
e) Move zone files from /named/master to /named/archive
# mv domain
starter removal[edit]
All starter sites on home-1.nethere.net
1) Removal primarily done via a NH script (webdel_home) on home-1
# nethere/sbin/webdel_home -h for usage i.e. webdel_home -p zn -s startername <== for removing znet
Disk Quota increases[edit]
Quotas are increased by using the "edquota" command:
- edquota username
Note that quotas are in KB, so a conversion is necessary. Also note that the "soft" quota is 1 MB less than the "hard" quota.
The calculations are as follows: "soft" = ( quota(in MB) - 1 ) x 1024 "hard" = quota(in MB) x 1024 e.g. for a 500 MB quota: soft => (500 - 1) x 1024 = 510976 hard => 500 x 1024 = 512000
To check a quota:
- quota -v username
You can also use the prototype users:
- edquota -p quota100 username (100 MB)
- edquota -p quota500 username (500 MB)
- edquota -p quota1000 username (1000 MB)
Granting of shell - Enterprise packages[edit]
Shell is granted by the "chsh" command. By default, C shell (csh) is used:
- chsh -s shell_needed username
e.g. # chsh -s csh username
- Note: customers must provide us with the static IP address(es) they will be connecting from. The IP address(es) must be added to the configuration on admin-1.
1) Modify the corresponding rules files on admin-1: /dist/files/nhweb/etc/ipf.rules.fxp0 , .em0 (RCS controlled) 2) Push updates to servers
- cd /dist/rdist
- gmake nhweb
- gmake nhweb-update
3) Reload the rules on the corresponding server that needs access to
- ipf -Fa -f /etc/ipf.rules
SSL certificate installs[edit]
1) Whois the site for information on Registrant, email - to be used for SSL generation
- whois domain.tld
2) Create SSL self signed certificate, get CSR for customer to sign
a) SSL generation is done by script: /nethere/conf/apache/pki/newsslcert.sh b) Copy CSR for customer
3) Create/update Apache configuration files via NH script (webadd_ssl)
# /nethere/sbin/webadd_ssl -h for usage a) webadd_ssl [-h] [-d domain] [-s ssl_domain] [-n] # webadd_ssl -d www.domain.tld -s www.domain.tld
4) Update DNS zone with new IP address - done on ns1.nethere.net
a) Check out DNS zone file in /named/master
# co -l domain.tld
b) Adjust A records:
--cut--
;; unixweb-## << enter the unixweb server number for ease of ID
@ IN A vhost_ip_address
www IN A vhost_ip_address
--cut--
c) Check in DNS zone file
# ci -u domain.tld < /dev/null
d) Reload zone file
# rndc reload domain.tld
e) check /var/log/named for errors
5) Update DNS PTR record for IP address - done on ns1.nethere.net
Note: Assuming IP address a.b.c.d
a) Check out in-addr.arpa zone for IP address in /named/master
# co -l a.b.c
b) Follow format for PTR records --cut--
d IN PTR www.domain.tld.
--cut--
c) Check in PTR zone file # ci -u a.b.c < /dev/null
d) Reload PTR zone
# rndc reload c.b.a.in-addr.arpa
e) check /var/log/named for errors
Updating SSL certificate[edit]
1) cd to /nethere/conf/apache/pki/ssl.crt on server site is hosted on
2) Check out (RCS) www.domain.tld.crt file
# co -l www.domain.tld.crt
3) Edit the file, remove old certificate, paste in new certificate
4) Check in (RCS) the www.domain.tld.crt file
# ci -u www.domain.tld.crt < /dev/null
5) Check, restart Apache
# apachectl configtest # apachectl stop # apachectl startssl
6) Verify httpd started:
# ps auxw | grep httpd
a) If no processes, will need to revert back to old SSL cert, and restart apache. Check logs for errors
# view /www/default/logs/ssl_engine.log
b) Check for "Unable to configure RSA server private key" and "key values mismatch" entries - this means a bad SSL certificate
Password protection[edit]
Http (simple) password protection is governed by the Apache configuration for the domain
1) Create userdb, users file in the domain root directory (/www/www.domain.tld):
# mkdir userdb # cd userdb # htpasswd -bc users username password
2) Check out Apache config for www.domain.tld in /nethere/conf/apache/conf/<vhosts,nvhosts>
# co -l www.domain.tld(.common)
3) Edit Apache configuration, add the following lines within the VirtualHost container --cut-- <Directory "/www/www.domain.tld/dir_to_be_protected">
AuthType Basic
AuthName "www.domain.tld/dir_to_be_protected authentication"
AuthUserFile /www/www.domain.tld/userdb/users
<Limit GET POST>
require valid-user
</Limit>
</Directory>
--cut--
4) Check in Apache config
# ci -u www.domain.tld(.common) < /dev/null
5) Restart Apache
# apachectl configtest # apachectl restart
More on http (simple) password protection can be found here: http://httpd.apache.org/docs/1.3/howto/auth.html#basic
.htaccess[edit]
Used if customers want control of certain Apache directives (i.e. Authentication, etc.)
1) Check out Apache config for www.domain.tld in /nethere/conf/apache/conf/<vhosts,nvhosts>
# co -l www.domain.tld(.common)
2) Add the AllowOverride directive in the <Directory> section, under the PHP FCGIWrapper
i.e. <Directory "/www/www.domain.tld/htdocs"> FCGIWrapper /www/www.domain.tld/htdocs/cgi-bin/php4 .php AllowOverride AuthConfig FileInfo Indexes Limit </Directory>
3) Check in Apache config
# ci -u www.domain.tld(.common) < /dev/null
4) Restart Apache
# apachectl configtest # apachectl restart
More info on Allow Override can be found here: http://httpd.apache.org/docs/1.3/mod/core.html#allowoverride
Domain re-provisioning[edit]
- NOTE: Following is for domain being provisioned on same server (i.e. just being renamed)
1) Adjust DNS - ns1.nethere.net
a) Rename the DNS zone file to the new domain, remove the old DNS zone files from /named/master and /named/master/RCS b) Check in the new DNS zone file c) Edit /named/named.master - replace the old domain with the new one, alias as necessary d) Reload DNS zones - make new-zone in /named
2) Adjust Apache config - server domain is hosted on
a) Find current config file(s) in /nethere/conf/apache/conf/<vhosts,nvhosts> b) Replace old domain name entries with new domain name, alias as necessary c) Save as new_domain.tld d) Remove old_domain.tld(.common), RCS/old_domain.tld(.common) e) Check in new_domain.tld via RCS:
# ci -u new_domain.tld < /dev/null
f) Check out Apache include configuration file via RCS /nethere/conf/apache/conf/<nvhosts.conf,vhosts.conf> g) Edit nvhosts.conf or vhosts.conf, replace old_domain.tld entries with new_domain.tld h) Check in via RCS /nethere/conf/apche/conf/<nvhosts.conf,vhosts.conf>
3) Rename directory for new domain
# cd /www # mv www.old_domain.tld www.new_domain.tld
4) Adjust PHP stub files/configuration
a) Adjust php.ini file in /www/www.domain.tld/(php4,php5) b) Adjust PHP stub files in /www/www.domain.tld/htdocs/cgi-bin/(php4,php5) *note: need to chflags to "noschg" for /www/www.domain.tld/htdocs/cgi-bin/(php4,php5) before being able to update the stub files after adjusting, be sure to chflags schg /www/www.domain.tld/htdocs/cgi-bin/(php4,php5)
5) Edit password file (vipw), replace old_domain.tld entries with new_domain.tld
# vipw
6) Restart Apache
# apachectl configtest # apachectl restart
- NOTE: Following is for domain being re-provisioned on new server
1) Run NH script "webadd" on sawfish to provision domain on new server, *DO NOT* reload DNS, use same user/pass as before. 2) After customer has uploaded site to new server and gives the ok do the following:
a) Update DNS zone for domain.tld; reload zone for domain.tld
b) Wait 48 hours, then remove the site off the old server
# /nethere/sbin/webdel
FTP space provisioning[edit]
Generally FTP sites are provisioned on the server that hosts the main website. In the case of NT based FTP sites, we usually provision them on the server with the most space available.
1) Provision site on server
a) Done via NH script (webadd_ftp): webadd_ftp [-h] [-d domain] [-u username] [-p password] [-n]
# /nethere/sbin/webadd_ftp -h <- for usage
EX: for ftp.domain.tld:
# /nethere/sbin/webadd_ftp -d ftp.domain.tld -u username -p password
(leaving off the -n will restart the proftpd process)
b) Note: The host IP address will be given when the provisioning is completed, use that ip (a.b.c.d) for DNS entries
c) Note: If this is an existing customer on the server, you'll need to increase the quota manually by 100 MB for the customer, see the section on quota increases for more info.
2) Adjust DNS for domain on ns1.nethere.net
a) Checkout zone for domain
b) Add ftp host entry for domain, adjust serial
EX for domain.tld:
--cut-- ftp IN A a.b.c.d --cut--
c) Check in zone for domain
d) Reload zone
# rndc reload domain.tld
3) Adjust PTR record for domain, adjust serial
a) Checkout zone for a.b.c b) Add record for domnain:
--cut-- d IN PTR ftp.domain.tld. --cut--
c) Check in zone for a.b.c
d) Reload zone
# rndc reload c.b.a.in-addr.arpa
4) Check for DNS errors
a) tail /var/log/namedb
SiteBuilder provisioning[edit]
unixweb-7.nethere.net
- Must re-provision site on unixweb-7.nethere.net (if not already done)
1) Log in to SB admin:
http://sitebuilder.nethere.net/admin a) username: root
2) Add site to SB config
a) Click on Site Management -> Add regular * Alias is website username: i.e. aa#### * Check the "Active" box * Plan is "BasePlan" * Password same as website b) Click on "Publish Properties * Check "Allow publishing" * Site host name: www.domain.tld * FTP host: unixweb-7.nethere.net * FTP login/password: same as site user/pass * FTP working directory: leave blank c) Click Apply
Name Servers[edit]
OS: FreeBSD
General[edit]
Nethere DNS has been migrated to PowerDNS on ganeti virtuals with a web admin here: https://nhdns.jcihosting.com/ Old instructions follow:
ns1.nethere.net - Primary name servers for DNS zone records
/named - contains the files that have all DNS domain zone entries (named.master, named.slave, named.acl) plus Makefile for distributing DNS records /named/master - contains all the domain zone files for which we are authorative for, as well as IP address (PTR records)
ns2.nethere.net - Secondary (slave) name server for DNS zone records
nsrbl-1.nethere.net - RBL (Realtime Blackhole List) DNS server
/named/rbldns/cache - contains the files for domains that we specifically allow or deny
nscache-1,2 - caching name servers
Adding DNS website entries[edit]
1) Create a DNS zone file for domain.tld via NH script (zoneadd_vhost)
# /nethere/sbin/zoneadd_vhost -h (for usage)
Adding DNS IP entries[edit]
1) Create the forward and reverse DNS records via NH script (zone_generate) for a netblock, will create /tmp/customer.forward and /tmp/customer.reverse files to be read
a) /nethere/sbin/zone_generate -h for usage
i.e. for netblock a.b.c, starting IP d, ending IP z
# zone_generate -n a.b.c -b d -e z -p customer
2) Checkout, edit the IP in-addr.arpa zone file, reload the zone
a) # co -l a.b.c
b) Seach for the nearest netblock area for the domain, follow format for customer info, read in the /tmp/abbrev.rdns file accordingly, increase Serial for zone in YYYYMMDD## format
e.g.
--cut--
- 66.63.152.232/30 (255.255.255.252)
- Description
- First Choice Home Improvement
- Contact
- Shannon Hill <firstchoicehi@hotmail.com>, (858) 277-5351
- Location
- AR-1, Serial3/0/18:0
232 IN PTR firstchoice-net.access.nethere.net. 233 IN PTR firstchoice-gw.access.nethere.net. 234 IN PTR firstchoice-2.access.nethere.net. 235 IN PTR firstchoice-bcast.access.nethere.net. --cut--
c) # ci -u a.b.c < /dev/null d) # rndc reload c.b.a.in-addr.arpa e) verify loading of zone: # tail /var/log/named
3) Checkout, edit the forward DNS zone file, reload the zone
a) # co -l access.nethere.net
b) Search for the nearest neblock area for the domain, follow format for customer, read in the /tmp/abbrev.fdns file accordingly, increase Serial for zone in YYYYMMDD## format
e.g.
--cut--
- 66.63.152.232/30 (255.255.255.252)
firstchoice-net IN A 66.63.152.232 firstchoice-gw IN A 66.63.152.233 firstchoice-2 IN A 66.63.152.234 firstchoice-bcast IN A 66.63.152.235 --cut--
c) # ci -u access.nethere.net < /dev/null d) # rndc reload access.nethere.net e) verify loading of zone: # tail /var/log/named
Unblocking RBLd IP addresses[edit]
There are two primary reasons why we add customers to the allow relay list: a) They have a static IP address (i.e. DSL, T1) and wish to use our mail servers b) They've been blocked by one of our subscribed blackhole lists, however, have patched their machine and are no longer open to relay.
To do this on nsrbl-1.nethere.net:
1) Checkout the allow.relays.nethere.net file located in /named/rbldns/cache
2) Edit the file, and add the IP address in the following format: --cut-- a.b.c.d YYYYMMDD hostname reason for listing --cut--
You can also add subnets via '/' notation for relay --cut-- a.b.c.d/28 YYYYMMDD hostname reason for listing --cut--
3) Check in the file
4) Updates to the rbldns zone are done automatically on the hour, every hour, so no need to do anything else.
Note:
For "permanent" (i.e. customer static IP addresses), add the IP address in the "## permanent allowed relay (i.e. customer w/ static IP)" section For "temporary" (i.e. blackhole listed IP addresses), add the IP address in the "## temporary" section
Also, for temporary IP addresses, we need to send the note to the requester detailing the following: a) If the IP address relays Spam/UCE/Viruses through us, it is to be removed permanently. b) The customer must follow the steps listed on the blackhole list the IP was listed on to get removed.
Flushing DNS cache for a domain[edit]
Due usually to a bad zone or excessively long TTL for a domain, the cache for it will need to be flushed.
On nscache-1:
- /usr/local/sbin/rndc flushname domain.tld
If cache is still corrupted (i.e. zone lookups either fail or are incorrect), need to stop and restart the caching server:
- /etc/init.d/local.named stop
- /etc/init.d/local.named start
SiteBuilder[edit]
http://sitebuilder.nethere.net/ -URL used to test sitebuilder
http://sitebuilder.nethere.net/Login -URL for control panel.
This is where the customer also logs in to manage their web site and also where you log in to administer sitebuilder. Sitebuilder is hosted on sb-2.nethere.net. The site is designed/built on this server and published to unixweb-7.nethere.net. This is the only server that can host a sitebuilder web site. The admin log in to manage sitebuilder is: Username: admin Password: N3tH3r31!
Cart32[edit]
cart32 information:
Cart32 is hosted on ntweb-6.nethere.net. That is where all of the configuration files are located. The location of the ini file is D:\websites\Cart32cgi/cart32.ini. This is where ip restrictions to admin panel are set, password can be reset for admin, time limit restriction reset, etc.
There are four customers that still use cart32:
https://www.cart.simplyweb.net/lab400/cart/c32web.exe https://www.cart.simplyweb.net/nutragenics/cart/c32web.exe https://www.cart.simplyweb.net/retrogen/cart/c32web.exe https://www.cart.simplyweb.net/stonesculptorssupplies/cart/c32web.exe
The client codes are lab400, nutragenics, retrogen, and stonesculptorssupplies. You can reset their passwords through the admin panel.
To administer cart32:
Control Panel: https://www.cart.simplyweb.net/cart/c32web.exe/Admin Username: administrator Password: N3tH3r31! Cart Admin Password: N3tH3r31!
Webmail[edit]
Removing webmail filters[edit]
All webmail filters are stored in the MySQL database on webmail-1.
Simple SQL commands:
- Choosing a DB
mysql> use db_name;
- Showing tables in a DB (must be using a DB)
mysql> show tables;
- Describing a table's fields
mysql> describe table_name;
For NetHere, the database is horde_nh.
For Simply, the database is horde_si.
For zNET, the database is horde_zn.
1) Log to MySQL as root, using the normal root password
a) # mysql -u root -p
2) Select the appropriate database
a) mysql> use horde_nh
3) Delete webmail filters.
a) mysql> delete from horde_prefs where pref_uid='<username>' and pref_name='filters';
Replace <username> with the users login.
Removing Address book entries[edit]
1) Log in 2) Select appropriate db 3) Find object_id, delete
a) mysql> select object_id from turba_objects where object_email='email@domain.tld'; b) mysql> delete from turba_objects where object_id='object_id_obtained_previously';
Repairing webmail address book[edit]
1) Export (via webmail, if possible) the address book to a csv file.
2) Delete the Turba objects in the mySQL database in the turba_objects table that is assigned to the customer.
mysql> delete from turba_objects where owner_id='username';
3) Import the saved address book via webmail.
Changing webmail From Information[edit]
1) Login as the customer (http://webmail.nethere.net)
2) Click on Options
3) Click on Personal Information
4) Select either default identity or a new one and click on Edit Your Identities
MySQL[edit]
All unixweb-## boxes have MySQL processes running on them. DB provisioning is done on the same server as which hosts the domain. For Windows 2000 MySQL provisioning, all DBs are placed on ntdb-2.nethere.net
MySQL DB provisioning[edit]
Use https://phpmyadmin.nethere.net User: root PW: <root db pw>
1) Select the server that the DB needs to be hosted on, typically the same server that the website is hosted on. 2) Create the DB
a) Under "MySQL" - "Create new database" - the DB name is typically the website username; click "Create"
3) Create the User, set permissions for user on DB
a) Click "Home" (upper left), then click "Privileges" (Under "MySQL")
b) Click "Add a new User" make sure "Any host" selected, "User name" is typically the website username, "Password" is typically the website password. !LEAVE "Global Privileges" settings deselected! Click "Go" (bottom right corner) when finished.
c) Under "Database-specific privileges" be sure to add the user database
d) Select DB, then for "Privileges:" make sure the following are checked:
Select, Insert, Update, Delete, Create, Alter, Index, Drop, Create Temporary Tables, Lock Tables, References
List Server[edit]
lists-1.nethere.net
Mail List provisioning[edit]
1. Go to the /mailman/bin directory: # cd /mailman/bin
2. Create the list on lists-1: # ./newlist -q \ listname@lists.domain.com \ mailman-owner@lists.domain.com \ password
3. Configure the list using default settings: # ./config_list -i /mailman/data/defaultlist.cfg listname
- IF A NEW DOMAIN FOLLOW INSTRUCTIONS BELOW ***
NOTE: All files in /etc/mail are controlled by RCS
4. Go to the /etc/mail directory: # cd /etc/mail
5. Add the list domain to the mailertable file: lists.domain.com mailman:lists.domain.com
6. Add the list domain to the relay-domains file.
7. Add the list domain to the virtuserdomain file.
8. Add the following entries to the virtusertable file: mailman@lists.domain.com <customer_email_address> mailman-owner@lists.domain.com mailman@lists.domain.com
9. Run make to rebuild all files: # make
10. Run make to restart sendmail: # make restart
Backup Server[edit]
backup-1 -> odd numberd unix backup-2 -> even numbered unix
On Saturdays, we do a level 0 dump
the rest are differential
Restoration fees are basically $25 per day restored for email, $150 for website from cancelled archive, $25 a day for website
Restoration is based on customer request. In order to restore a site/mailbox fully, you must start restoration from the previous level 0 backup and continue until the day after the day requested, since backups are performed in the morning.
General rule is to leave the gzipped file available for 24 hours, after that remove it.
Site Restoral[edit]
If the site was removed by the web_del script, then so long as it is within 30 days since removal, the gzipped site will be available within /www/archive/www.domain.tld-date_removed.tar.gz Extract with tar:
- tar -zxvpf www.domain.tld-date_removed.tar.gz
If the site needs to be restored from backup, here are the steps
1) To determine where backups are stored, go to /nethere/conf/backup
2) grep for the server within the directory to determine the backup directory used i.e.
- grep unixweb-1 *
4) Go to /backup1/unix/unixweb-1.nethere.net
a) you'll see a bunch of directories, named <date>-<dump_level>
5) Since the backups happen in the morning, typically you'll need to go to the day before, restore
e.g. for 11/10
a) cd 20041109-3
b) restore -if and whatever the filesystem is named
(for interactive: add files/dirs, extract, then use 1 for volume #)
i.e.
restore> add <path_to_dir/files>
restore> extract
(for volume #: 1)
(Set owner: y)
6) After you've restored the files, just use scp as root on sawfish to copy them off of backup-# and then onto unixweb-#
a) (on sawfish - two step process)
# scp backup-1:/tmp/<restored_file> /tmp
# scp /tmp/<restored_files> unixweb-1:/tmp
7) Cleanup any restored files off of backup-# server and sawfish
With the newly restored files, following are the steps necessary to restore a site:
1) Restore www.domain.tld directory to /www
# mv /www/archive/www.domain.tld /www
2) Restore configuration files
a) Apache: mv /www/archive/nethere/conf/apache/(n)vhosts/www.domain.tld /nethere/conf/apache/conf/(n)vhosts/ b) Webalizer: mv /www/archive/nethere/conf/webalizer/unix/www.domain.tld /nethere/conf/webalizer/unix
3) Add user to /etc/master.passwd:
a) vipw
(go to end of file, read in the master.passwd file from /www/www.domain.tld)
:r /www/www.domain.tld/master.passwd
b) Remove /www/www.domain.tld/master.passwd
4) Change flags on cgi-bin/ directories:
# chflags sunlnk /www/www.domain.tld/htdocs/cgi-bin/ # chflags schg /www/www.domain.tld/htdocs/cgi-bin/php*
5) Add configuration file entry to nvhosts/vhosts.conf
# co -l (n)vhosts.conf # vi (n)vhosts.conf # ci -u (n)vhosts.conf
6) Check apache, restart; verify apache
# apachectl configtest # apachectl graceful #
7) Cleanup remaining restored files
# rm -r /www/archive/www # rm -r /www/archive/nethere
8) Enter zone into DNS on ns1.nethere.net
a) zone file
# mv /named/archive/domain.tld /named/master
# mv /named/archive/domain.tld,v /named/master/RCS
b) named.master file
# co -l /named/named.master
(add zone)
# ci -u /named/named.master
# cd /named; make new-zone
# tail /var/log/named (look for errors)
(12:40:47 PM) Henry Chan: restore is now available that is compatible with the 4.4bsd format (12:41:04 PM) Henry Chan: to restore, use the following command: restore -c -i -f path_to_archive (12:41:08 PM) Henry Chan: the "-c" is what does it (12:41:29 PM) Henry Chan: (only applies to backup-1-new... doesn't work on backup-2 or backup-4)
Email Restoral[edit]
Storage path prefixes:
/nfs/1 is sndg-netapp-1 (on backup-4 - /backup/hosts/2/e0.sndg-netapp-1.nethere.net)
/nfs/2 is sndg-netapp-2 (on backup-3 - /backup/hosts/2/e0.sndg-netapp-2.nethere.net)
/nfs/3 is sndg-netapp-3 (on backup-2 - /backup/hosts/2/e0.sndg-netapp-3.nethere.net)
/nfs/4 is sndg-netapp-1 (on backup-2 - /backup/hosts/2/e0.sndg-netapp-1.nethere.net)
/nfs/5 is sndg-netapp-2 (on backup-1 - /backup/hosts/2/e0.sndg-netapp-2.nethere.net)
/nfs/6 is sndg-netapp-3 (on backup-2 - /backup/hosts/2/e0.sndg-netapp-3.nethere.net)
Looking up a mailbox's Storage Path:
You will need to know which directory and NFS server the customer's mail is
stored on. Use the Provisioning Tool to get the Storage Path (find the customer's mail
account, then click on the Engineering sub-tab under the Email tab). You should end up
with something like "/nfs/3/nh/h/t/htchan/Maildir". Match the prefix of the storage
directory with an NFS server (see prefixes above).
1) If restoring a recently deleted mail account:
a) Look for the archive on mailbox-1 in /nfs/archive/mail/{platform}/{username}.{date}-{PID}.tar.gz.
If it doesn't exist, it's been too long and the only way to get email back is to restore from backup.
b) Make sure the account is re-created in Provisioning Tool and look up the storage path
(see "Looking up a mailbox's Storage Path" above).
c) Extract the archive to a temporary directory:
Sample command:
cd /tmp; tar xzvpf /nfs/archive/mail/nh/zella.20090902-7508.tar.gz
Sample output:
nfs/2/nh/z/e/zella/
nfs/2/nh/z/e/zella/Maildir/
nfs/2/nh/z/e/zella/Maildir/tmp/
nfs/2/nh/z/e/zella/Maildir/new/
nfs/2/nh/z/e/zella/Maildir/cur/
nfs/2/nh/z/e/zella/Maildir/maildirsize
nfs/2/nh/z/e/zella/Maildir/.Trash/
nfs/2/nh/z/e/zella/Maildir/.Trash/tmp/
nfs/2/nh/z/e/zella/Maildir/.Trash/new/
nfs/2/nh/z/e/zella/Maildir/.Trash/cur/
nfs/2/nh/z/e/zella/Maildir/.Trash/maildirfolder
nfs/2/nh/z/e/zella/Maildir/.Drafts/
nfs/2/nh/z/e/zella/Maildir/.Drafts/tmp/
nfs/2/nh/z/e/zella/Maildir/.Drafts/new/
nfs/2/nh/z/e/zella/Maildir/.Drafts/cur/
nfs/2/nh/z/e/zella/Maildir/.Drafts/maildirfolder
nfs/2/nh/z/e/zella/Maildir/.Sent Items/
nfs/2/nh/z/e/zella/Maildir/.Sent Items/tmp/
nfs/2/nh/z/e/zella/Maildir/.Sent Items/new/
nfs/2/nh/z/e/zella/Maildir/.Sent Items/cur/
nfs/2/nh/z/e/zella/Maildir/.Sent Items/maildirfolder
nfs/2/nh/z/e/zella/Maildir/courierpop3dsizelist
Note:
In the above output, notice that the Maildir is located in "nfs/2/nh/z/e/zella", it will be used in the next step.
d) Copy the Maildir files to the new mailbox storage path using tar (tar does a better job at preserving
things like symbolic links and permissions than cp or mv):
Sample command:
cd /tmp/nfs/2/nh/z/e/zella; tar cf - Maildir | (cd /nfs/3/nh/z/e/zella; tar xvpf -)
Output:
You should list the files as it is copied. Basically, similar to what you
saw in step "c" above but without the "nfs/2/nh/z/e/zella" prefix.
Note:
The command format is basically:
cd /tmp/{restored-maildir-path-see-note-in-step-c}; tar cf - Maildir | (cd {new-storage-path-without-Maildir}; tar xvpf -)
e) Clean up temporary directory:
Sample command:
cd /tmp; rm -rf nfs
2) If restoring a zfs mailbox (from the last week) to a particular date's backup:
a) login to the correct sndg-netapp-[1/2/3]-new
b) get the files from the correct zfs snapshot
Sample commands:
cd /tank0/mail/.zfs/snapshot/20180215-0/nh/c/h/christineat
ls -l
cd to the desired directory and copy the files over to the users mailbox
cp -p /tank0/mail/.zfs/20180215-0/nh/c/h/christineat/Maildir/cur /tank0/mail/nh/c/h/christineat/Maildir/cur
3) If restoring a mailbox to a particular date's backup:
a) Look up the storage path to the customer's mailbox and determine which backup server the
dump file is on (see "Looking up a mailbox's Storage Path" and "Storage path prefixes" above).
b) Restore the level 0 and any incremental backups (in order) to /tmp on the backup server:
Sample commands:
cd /tmp
restore4x -if /backup/hosts/2/e0.sndg-netapp-2.nethere.net/20090912-0/mail.dump
chflags -R 0 /tmp
restore4x -if /backup/hosts/2/e0.sndg-netapp-2.nethere.net/20090913-1/mail.dump
chflags -R 0 /tmp
...skipped repetitive stuff here...
restore4x -if /backup/hosts/2/e0.sndg-netapp-2.nethere.net/20090918-6/mail.dump
chflags -R 0 /tmp
Note:
After each restore, we need to recursively remove all flags from /tmp to eliminate
the immutable flag that gets set on the files (this happens only on NetApp dumps... who knows why).
c) Create a new tar of the Maildir directory so that it can be copied to mailbox-1 for further processing:
Sample command:
cd /tmp/nh/z/e/zella; tar cf /tmp/archive.tar Maildir
d) Clean up /tmp:
Sample command:
rm -rf /tmp/nh
Note:
Depending on the platform, it might be /tmp/nh, /tmp/si, or /tmp/zn.
e) Copy the /tmp/archive.tar file to /tmp on mailbox-1.
f) On mailbox-1, extract the Maildir archive on top of what they already have:
Sample command:
cd /nfs/2/nh/z/e/zella; tar xvpf /tmp/archive.tar
g) Remove /tmp/archive.tar.
Beginning of old instructions[edit]
Use the same techniques as site restoration, with the following exceptions:
1) Since everything is in maildir format, you have to reassmble things in /tmp/<username>, tar it up, and then restore it on any of the mailbox machines
a) using the following syntax:
# tar -zcpf /tmp/username.tgz yyyymmdd-#/platform yyyymmdd-#/platform yyyymmdd-#/platform
i.e.
# tar -zcpf /tmp/username.tgz 20050915-5/nh 20050914-4/nh 20050910-0/nh
2) In regards to mail spools, you'll need to use the chflags command to adjust the flags on the files. For whatever reason, they are stored with a system immutable flag
a) Do chflags -R noschg <dir> on the restored directory before copying the files with sawfish otherwise, the files cannot be deleted:
i.e.
# chflags -R noschg /backup1/unix/sndg-netapp-1-e2b.nethere.net/20050105-4/
b) After tar/gzipping, remove the restored directories
i.e
# rm -r /backup1/unix/sndg-netapp-1-e2b.nethere.net/20050105-4/nh
3) Can restore quickly with script on mailbox-1 (or mailbox-2) using the NH script "restoremail"
a) # /nethere/sbin/restoremail -h for usage
i.e.
# restoremail -p nh -f nhusername.tgz -n 2 -u nhusername
- End of old instructions ***
- Backup locations *
backup-1: backup-3.nethere.net fpweb-1.nethere.net home-1.nethere.net koi.nethere.net marmaduke.inetworld.net ntdb-1.nethere.net ntweb-1.nethere.net ntweb-3.nethere.net ntweb-5.nethere.net ntweb-7.nethere.net phoenix.nethere.net unixweb-1.nethere.net unixweb-3.nethere.net unixweb-5.nethere.net unixweb-7.nethere.net shark.nethere.net tetra.nethere.net wms-1.nethere.net
backup-2: andromeda.nethere.net backup-4.nethere.net eel.nethere.net fpweb-2.nethere.net ntdb-2.nethere.net ntweb-2.nethere.net ntweb-4.nethere.net ntweb-6.nethere.net unixweb-2.nethere.net unixweb-4.nethere.net unixweb-6.nethere.net ds.znet.com mx1.znet.com mx2.znet.com mx3.znet.com dmx.znet.com la.znet.com uf.znet.com
backup-3: backup-1.nethere.net lists-1.nethere.net mailbox-1.mail.nethere.net mailbox-3.mail.nethere.net mta-1.mail.nethere.net mx-1.nethere.net nscache-1.nethere.net nsrbl-1.nethere.net pegasus.nethere.net scan-1.mail.nethere.net sndg-netapp-2.nethere.net relay-1.mail.nethere.net webmail-1.mail.nethere.net
backup-4: ahi.nethere.net backup-2.nethere.net cp-1.nethere.net cygnus.nethere.net dragon.nethere.net lisa.nethere.net mailbox-2.mail.nethere.net mta-2.mail.nethere.net news-1.nethere.net nscache-2.nethere.net nsrbl-2.nethere.net sawfish.nethere.net scan-2.mail.nethere.net scribe.nethere.net sndg-netapp-1.nethere.net relay-2.mail.nethere.net urchin.nethere.net
Mail servers[edit]
Organization[edit]
NetHere Mail Servers
====================
Customers Inbound Mail Outbound Mail
/\ || || /\
|| || || smtp ||
|| || \/ ||
|| || ServerIron ||
|| || || ||
|| || || ||
|| || smtp \/ ||
|| || =========> mta-1 mta-2 ====> relay-1 <==== unixweb-* servers
|| || || /\ relay-2 ntweb servers
http || pop3 || || ||
|| imap || \/ +--------------+
|| || ServerIron |
|| || || |
|| || || |
|| || \/ |
|| || nsrbl-1 nsrbl-2 |
|| || (spamhaus, dcc blacklist) |
|| || || |
|| \/ \/ |
\/ mailbox-1 <=== scan-1 scan-2 |
webmail <===> mailbox-2 (sendmail/amavisd/sophos) |
imap mailbox-3 |
/\ /\ |
|| || \/
nfs || +---------------------------> mta-db
|| (LDAP)
\/
sndg-netapp-1
sndg-netapp-2
sndg-netapp-3
Overview[edit]
mta-1.mail.nethere.net- Primary servers for all inbound and outbound mail. mta-1.mail.nethere.net- Primary servers for all inbound and outbound mail.
nsrbl-1.nethere.net- Realtime blacklist check using dccd and spamhaus. The mta servers query the ip address of the mail server against the blacklist before transferring the email to the scanning servers.
nsrbl-2.nethere.net
scan-2.mail.nethere.net- Scans email with Sophos (currently disabled) and Spam Assasin before sending it to the mailbox servers. scan-2.mail.nethere.net Scans email with Sophos (currently disabled) and Spam Assasin before sending it to the mailbox servers.
mailbox-1.mail.nethere.net- Front end server for the mail directories. Queries LDAP server for miscellaneous customer information including the directory the email should be stored in, passwords, spam sensitivity levels, etc. before delivering email to appropriate directory. mailbox-2.mail.nethere.net mailbox-3.mail.nethere.net
Mail is stored in maildir format:
/nfs/<nfs_number>/<platform>/u/s/username
sndg-netapp-1-new.nethere.net- Solaris system using zfs set up as a nfs mount on mailbox-1, -2, and -3. Email is stored on these platforms for all customers. sndg-netapp-2-new.nethere.net sndg-netapp-3-new.nethere.net
mx-1.nethere.net- spooling mail server
mailx-1.nethere.net- all outbound mails originating from a web server is sent out this email server. All web servers are set to use mailx.nethere.net which resolves (through the serveriron) to either mailx-1.nethere.net and mailx-2.nethere.net. Since we shut down mailx-2, all outbound mail from the web servers goes through mailx-1. This was to prevent all the customers email from being blacklisted as a result of a web site being compromised.
mta-db.mail.nethere.net- database server for the mta servers. All postfix databases are stored here.
webmail.nethere.net- Hosts the webmail program for webmail.nethere.net, webmail.simplyweb.net and webmail.znet.net.
Client mail settings[edit]
Inbound Mail Server:
POP3 pop3.nethere.net pop3.znet.net pop3.simplyweb.net
IMAP imap.nethere.net imap.znet.net imap.simplyweb.net
Outbound Mail server:
mail.nethere.net mail.znet.net mail.simplyweb.net
Webmail:
webmail.nethere.net webmail.znet.net webmail.simplyweb.net
Control Panel:
cp.nethere.net cp.znet.net cp.simplyweb.net
nsrbl Disk Quota Warning[edit]
In the event you get a nagios regarding space on the two nsrbl servers, it is most likely the result of the dccd databases getting too large. To clear out space, log into each nsrbl server and execute the following:
df -h
cd /var/dcc/libexec
./stop-dccd
cd /var/dcc
rm dcc_db*
sync ; sync
cd /var/dcc/libexec
./start-dccd
ps auxw | grep dccd
Update outbound quota for a single ip address[edit]
We restrict the total number of emails that a customer can send to 2000 email recepients per 24 hour period. This is tracked by the ip address of the computer that connects to the mta server when it sends an outbound emal. The ip address and count is stored in the postfix database server, mta-db.mail.nethere.net. To increase the quota for a single ip address, you can log into either mta-1, mta-2, or console direcly into the database server. To use one of the mta servers:
1. Log onto mta-1.mail 2. Switch to mysql $mysql -h 10.0.0.69 -u policyd_outbound -p username: policyd_outbound password: p0stf1x! 3. Review database information mysql>show databases; 4. Switch to the policyd table. mysql>use policyd; 5. Review table header information mysql>show tables; mysql>describe throttle; 6. Verify the ip address is above quota mysql>select * from throttle where _from='xxx.xxx.xxx.xxx'; #this is the ip address of the account you are increasing. 7. Increase the maxium number of recepients the ip address can send to mysql>update throttle SET _rcpt_max = _rcpt_max + 20000 where _from='24.249.205.66'; 8. Verify mysql>select * from throttle where _from='xxx.xxx.xxx.xxx';
Remove a blocked mail server[edit]
The message in maillog helps identify where it is blocked.
Relaying denied due to excessive spam admin-2:/dist/files/nhmta/nethere/conf/postfix/common/client_reject Sender address rejected: Relaying denied due to Spam admin-2:/dist/files/nhmta/nethere/conf/postfix/common/sender_reject Relaying denied due to SPAM policyd??? Client host rejected: ... listed at zen.spamhaus.org=127.0.0.2 nsrbl-1:/named/rbldns/zones/*.spamhaus.org status=sent (250 2.7.1 Ok, discarded, id=95886-33 - SPAM) user mail protection in provisioning????
Remove a mail server from policyd blacklist[edit]
The ip address of a mail server can be placed on a blacklist for several reasons, to high of rate of inbound email from a single ip address, incorrectly formatted helo, etc. In the event you need to remove a mail server's ip from the policyd blacklist, you can log into either mta server of connect directly to the policyd database server, mta-db.mail.nethere.net. To remove the ip address using a mta server:
1. Log onto mta-1.mail 2. Switch to mysql $mysql -h 10.0.0.69 -u policyd_inbound -p policyd username: policyd_inbound password: p0stf1x! 3. Display database information mysql>show databases; 4. Use policyd database mysql>use policyd; 4. Display table information mysql>show tables; mysql>describe blacklist; 5. Verify ip address has been blacklisted mysql>select * from blacklist where _blacklist='xxx.xxx.xxx.xxx'; 6. Delete the entry mysql>delete from blacklist where _blacklist='xxx.xxx.xxx.xxx'; 7. Verify mysql> select * from blacklist where _blacklist='xxx.xxx.xxx.xxx';
The suspected mail server may also be in the helo table.
mysql> describe helo ; 1. Check the IP address mysql> select * from helo where _host='xxx.xxx.xxx.xxx' ; 2. Check the server helo name mysql> select * from helo where _helo like '%server%' ; 3. Delete the entry mysql> delete from helo where _host='xxx.xxx.xxx.xxx' ; mysql> delete from helo where _helo like '%server%' ;
Remove a mail server from spamhaus blacklist[edit]
ssh to nsrbl-1.mail and nsrbl-2.mail
cd /named/rbldns/zones find which file the IP Address is in. grep <IP Addr> *.spamhaus.org co -l <xxx.spamhaus.org> vi xxx.spamhaus.org ci -u <xxx.spamhaus.org>
Add ip address to client_reject[edit]
The client_reject list is one of the ways we combat spam. If we receive spam compaints from a mail server or otherwise identitfy a mail server as a source of spam, we can add it to a client_reject list. An email sent from a mail server on the client_reject list is rejected with a notice saying "Relaying denied due to excessive spam". To add an ip address to the list, we use admin-2. Once the list is updated on admin-2, it is pushed out to mta-1 and mta-2. To add an ip address on admin-2:
Log into admin-2.nethere.net. bash-3.00# cd /dist/files/nhmta/nethere/conf/postfix/common bash-3.00# co -l client_reject RCS/client_reject,v --> client_reject bash-3.00# vi client_reject -Add the ip address to the bottom of the list in the appropriate format. It you want to reject any email from the mail server, you would use: xxx.xxx.xxx REJECT Relaying denied due to excessive spam where xxx.xxx.xxx is the /24 subnet of the mail server the spam originated from. Do not place a period after the last octet. For example, to add a /16 subnet you would use: xxx.xxx REJECT Relaying denied due to excessive spam The REJECT key word tells postfix to reject the email. If you want to allow email from a /24 subnet, use the following format: xxx.xxx.xxx OK This tells postfix to accept email from that subnet. bash-3.00# ci -u client_reject RCS/client_reject,v <-- client_reject new revision: 1.286; previous revision: 1.285 enter log message, terminated with single '.' or end of file: >> . done bash-3.00# make /nethere/software/nhmta/postfix-2.4.5/sbin/postmap -C . client_reject mv client_reject.db maps/hash/client_reject.db bash-3.00# cd /dist/rdist bash-3.00# gmake nhmta-update updating host mta-1.mail.nethere.net updating: /dist/files/nhmta//nethere/conf/postfix/common/maps/hash/client_reject.db updating: /dist/files/nhmta//nethere/conf/postfix/common/client_reject updating host mta-2.mail.nethere.net updating: /dist/files/nhmta//nethere/conf/postfix/common/maps/hash/client_reject.db updating: /dist/files/nhmta//nethere/conf/postfix/common/client_reject bash-3.00#
Free mysql disk space on mta-db.mail.nethere.net[edit]
In response to nagios alert "[Nagios] PROBLEM alert - mta-db.mail.nethere.net/Disk mysql is WARNING", to clear space within the mysql database:
1. ssh to mta-db.mail.nethere.net.
2. Log into mysql and execute the following:
[root@mta-db user]# mysql -uroot -p
Enter password: dB@dm1N!
mysql> show databases;
mysql> use policyd;
mysql> show tables;
mysql> describe helo;
mysql> describe throttle;
mysql> describe throttle_from_instance;
mysql> select * from throttle_from_instance limit 10;
mysql> select now();
mysql> select unix_timestamp(now());
mysql> select count(*) from throttle_from_instance where _expire > 1426534697;
mysql> select max(_expire) from throttle_from_instance;
mysql> truncate table throttle_from_instance;
mysql> select count(*) from throttle_from_instance;
mysql> select * from throttle_from_instance;
mysql> show tables;
mysql> optimize table throttle_from_instance;
mysql> show databases;
mysql> use policyd;
mysql> quit
[root@mta-db user]# df -h
Errors[edit]
Common webmail errors[edit]
Error: --cut-- Fatal error: Call to undefined function: applicatio€”¬p() in /www/webmail.nethere.net/htdocs/x/m/templates/message/navbar.inc on line 7 --cut--
Solution: Restart apache on webmail-1 (apachectl restart)
Error:
--cut--
ERROR There was an error sending your message: unable to add recipient
[webhosting@nethere.com]: Invalid response code received from server
--cut--
Solution: More than likely DNS/domain issue, but check mta-1.nethere.net to be sure: --cut-- Aug 30 15:08:54 mta-1 nh/smtpd[50501]: NOQUEUE: reject: RCPT from webmail-1.mail.nethere.net[66.63.128.181]: 450 <orders@mulligrins.com>: Sender address rejected: Domain not found; from=<orders@mulligrins.com> to=<webhosting@nethere.com> proto=ESMTP helo=<webmail.nethere.net> --cut-- Problem: Webmail shows "1-5" messages but inbox appears empty. POP shows messages #1,2,3,4,5... could not be retrieved... server response error cannot open the message file it's gone.
Solution: Ownership issue
1) Use the provisioning tool to determine mailbox location:
a) Click on email account, Engineering b) Location is the "Storage Path:"
2) Log in to either mail server then change ownership of maildir to mailuser:mailuser
# chown -R mailuser:mailuser /nfs/#/platform/u/s/username
DEPRECIATED INFORMATION[edit]
Webservers[edit]
Allowing SSI (Server Side Includes)[edit]
- NOTE: This has been DEPRECIATED, all servers now allow SSI by default *****
We do not allow EXEC permissions for SSI.
1) Check out Apache config for www.domain.tld in /nethere/conf/apache/conf/<vhosts,nvhosts>
# co -l www.domain.tld(.common)
2) Add the following lines under the <Directory> directive --cut-- AddType text/html .shtml AddHandler server-parsed .shtml --cut--
3) Edit the "Options" line to be the following: --cut-- Options Indexes FollowSymLinks IncludesNOEXEC --cut--
4) Add a DirectoryIndex line above the <Directory> directive with the following: --cut-- DirectoryIndex index.shtml index.html index.htm home.html home.shtml index.php --cut--
- In summary, the config should look something similar to this **
--cut-- DocumentRoot "/www/www.domain.tld/htdocs"
DirectoryIndex index.shtml index.html index.htm home.html home.shtml <Directory "/www/www.domain.tld/htdocs"> AddType text/html .shtml AddHandler server-parsed .shtml Options Indexes FollowSymLinks IncludesNOEXEC Order allow,deny Allow from all </Directory>
--cut--
5) Check in Apache config
# ci -u www.domain.tld(.common) < /dev/null
6) Restart Apache
# apachectl configtest # apachectl restart
More SSI notes can be found here: http://httpd.apache.org/docs/1.3/howto/ssi.html
SimplyInternet[edit]
Z-net[edit]
Migrating to LAMP Host Servers[edit]
Below are outlines of the steps to migrate website & email hosting from nethere to lamphost configured servers. This is almost certainly incomplete at this stage. The exact steps to follow can vary depending on specifics of the client, so use this as a guide only. Be ready to adjust commands or add steps as needed.
Edit Nethere DNS[edit]
ssh user@admin-1.nethere.net
sudo -i
ssh ns1.nethere.net
cd /named/master
co -l example.com
vi example.com
ci -u example.com
rndc reload example.com
Make sure to increment serial number by 1 each time a change is made.
Migrate Website to nh5.jcihosting.net[edit]
Set TTL to 300 on ns1.nethere.net
Open up these web admins to get current account/site info:
http://toolbox.nethere.net/ -> Provisioning Tool (search for domain name)
https://secure.johncompanies.com/mgmt/index.html (search for col#####)
1. Create the lamphost user.
https://www.lamphost.com/admin/user/user/create
username: col#####
email: same as listed for col##### account
password: random, secure, won't be shared with client
defaults on rest of page
2. Create the customer account
https://nh5.jcihosting.net:8443/
LAMP Host Users -> Add User -> col#####
Virtual Hosts -> Add Virtual Host
Username: Select same username as above from pulldown
Domain: example.com
Hosting Plan: select plan that matches what's listed in https://secure.johncompanies.com/mgmt/view.html?cid=col#####
Add default DNS: No
3. Virtual Hosts -> Site Manager (next to domain added)
Site Manager Users -> Add User
Username: can be anything - I typically use Optigold Login from nethere admin
Password: make up something secure and record - this one will go to the user
4. Email & FTP -> Add Account
Email Account: ftp
Password: make up something secure and record - this one will go to the user
Enable FTP Access: Yes
5. Optional: Add MySQL database if site needs it (WordPress etc)
ssh to nh5.jcihosting.net and run 'add_mysql.php <example.com>'
MySQL database info will be emailed to webmaster@lamphost.com (and output on command line - ignore instructions to run additional commands)
https://mail.lamphost.com Sm77DdnQ
6. Copy over website
Login via ssh nh5
ssh YOURLOGIN@nat.johncompanies.com ssh root@nh5.jcihosting.net
cd /var/www/example.com/
rsync -v --archive --one-file-system --delete --delete-during --rsh=/usr/bin/ssh dsmith@unixweb-10.nethere.net:/www/www.example.com/htdocs ./
Change 'matt' to your username. Note that I had to add my username to groups httpd & webuser to allow me to copy website files without being root.
chown -R col#####:col##### htdocs
7. If site has MySQL data find the connection info. For example, WP sites will have the info in htdocs/wp-config.php
mysqldump -h mysqldb-1.webhost.nethere.net -u username --password=password databasename > databse.sql
mysql exampledotcom < database.sql
rm database.sql
8. If site is ready to be made live on nh5.lamphost.net then update DNS on ns1.nethere.com
Send email to client with details. Search support@jcihosting.com Sent folder for emails with Subject "updated hosting for" for example emails. You will have to adjust as needed (username/passwords, different info based on specific client).
9. After you finish, update the johncompanies.com backend for the col0# CID and change the nethere hosting server to nh3.jcihosting.net.
10. To add a SSL certificate
letsencrypt-create.php <domain.com>
To remove a SSL certificate
del-letsencrypt-certificate.php <domain.com>
Migrate Email to nh5.jcihosting.net[edit]
If migrating website & emails both follow the above steps to transfer the website first.
If migrating emails only the follow the first steps 1-3 of the above steps to set up the virtualhost on nh3.
1. Set TTL for MX record to 300.
2. Generate email report:
Log in to http://toolbox.nethere.net/
Select Provisioning Tool and search for the domain
Navigate to the Email section and then Generate Report.
Select these fields for the report:
Mail address
Alias
Forwarding address
Mail storage path
Copy the result of the report and save it as a text file in /tmp/ on nh5.jcihosting.net named example.com.txt (replace domain name)
3. Create the e-mail accounts
Make sure the virtualhost has enough email accounts allocated before proceeding. Edit the virtualhosts.virtualhost_settings table directly if need be to increase allocation.
Log in here: https://nh5.jcihosting.net:8443/ Select "MySQL.LH" Go to the "virtualhosts" database Search the "virtualhost_settings" settings for 'example.com' and adjust the 'email_accounts' setting.
Run this script once and only once:
/usr/local/bin/migrate_mail_accounts_from_nethere.php example.com
That will create all email accounts and aliases.
A file containing the list of email accounts and their new passwords will be created as /tmp/example.com-passwords.txt. Grab a copy of that for sending to the client.
4. Sync the emails
Another file will be created that contains the bash commands to sync emails from the old server. This will be named /tmp/example.com-sync.sh
Copy the file somewhere like the /root/ home dir and add "#!/bin/bash" as the first line and change perms to 750. Now you can run this script to sync all mails from nethere to nh3. Run it immediately to get an initial sync of emails.
When ready to switch email hosting over run the sync script one more time, then update DNS on nh3.nethere.net.
If you'd like to sync again after the move (if there was a delay or you think there may have been incoming emails during the transtion) remove the '--del' option from the sync script and then re-run.
5. Send email to customer
Search support@jcihosting.com Sent folder for messages with Subject "new email hosting for" for examples of emails to use for communicating with the client.
6. Update DNS and put NetHere mail on hold Once DNS has been pointed to the new server, go into the NetHere Toolbox http://toolbox.nethere.net/ and go to Domain -> General and then set Status to 'On Hold'.
7. Update host in JohnCompanies database.
Moving between Host Servers on Lamphost[edit]
You should delete the old virtual host before creating the new virtual host. It may take up to an hour to delete the old virtual host. To speed up the delete:
/opt/lamphost/cron-bin/del_virtualhosts.php
If you create the new virtual host before, you will need to update the mylampsite.com:
/opt/lamphost/bin/update_mylampsite_subdomain.php databaseproviders.com
Upgrade a Lamphost server to increase database limit[edit]
I've increased the database limit for this customer to 2. Now they can add a second db via the Site Manager admin.
Here is how I made the adjustment (we don't have a friendly admin for this function): Log in to Server Manager: https://nh3.jcihosting.net:8443/ Select the "MySQL.LH" phpMyAdmin Select 'virtualhosts' db. Adjust the 'databases' setting for the argee.com domain in the 'virtualhost_settings' table.
looking in the mail log files[edit]
exigrep