Category Archives: Linux HOWTO

Linux HOWTO

Permanently Ban Those Caught By Fail2Ban

Leave a comment

Introduction

Fail2ban is probably one of the best intrusive detection based tools an administrator can deploy onto their system. This is especially the case if your system is connected to the internet. If you aren’t already using it; consider reading my blog entry here that talks about it.

In this blog, I provide a scripted solution that will capture the current list of banned users from Fail2Ban and make their ban permanent. This allows us to limit the constant emails one might receive about the same people trying to compromise our system(s). For those who aren’t using the emailing portion of Fail2Ban; this script still greatly takes the load off of Fail2Ban because it no longer has to manage the constant repeat offenders. Our logs are less cluttered too.

The script will address several things:

  1. It will handle multiple attacks from people within the same Class C type netmask.
  2. It will allow for the permanent bans to stick even after your system is rebooted. Unlike Fail2Ban’s list of blocked perpetrators, which is lost if the system (or iptables) is restarted.
  3. It will enforce the use of iptable’s DROP directive instead of REJECT. This is a slightly more secure approach in handling those who aren’t ever allowed to come back.
  4. It will support the fact that over time, you may want to add and remove from this new ban list I keep speaking of. Basically you can re-run the steps outlined in this blog again (and again) and not lose the addresses you’ve already blocked.
  5. The script maintains a global list of addresses in a simple delimited format. You can then choose to share this list with other systems (or colleagues) to block the same unwanted users on these systems too.

Script Dependencies and Requirements

For this script to work, you can virtually use any Linux/FreeBSD/Unix operating system as long as you’re also using fail2ban in conjunction with iptables.

The script makes use of sed, and gawk to massage the data. These tools are common an available to all operating systems. But not all of them necessarily have them installed by default. Make sure you’ve got them before proceeding:

# Fedora / CentOS 4,5,6 / RedHat 4,5,6
yum -i install gawk sed

# Ubuntu / Debian
apt-get install gawk sed

The Goods

Without further ado, the below code is documented quite heavily so that you can just copy the sections into your terminal screen as the systems root user. Don’t try the next section until you’re done with the previous.

Although this code works for me, I still must caution you all the same: I will not be held liable for any loss, damage or expense as a result of the actions taken by my script. I’ve only used it without problem with CentOS 6.x. That said, the simplicity of it should make it work with any other *nux based OS as well.

# Author: Chris Caron
# Date: Tue, Apr 7th, 2015
########################################
# Environment Variables
########################################
# YOU MUST CORRECTLY SET THESE OR THE REMAINING FUNCTIONS OF
# THIS CODE WILL NOT WORK CORRECTLY.
#
# THIS SCRIPT ASSUMES YOU ARE RUNNING AS 'root'
#
# Public Ethernet Device
# For my home system, i have this set to ppp0; You'll want to
# set this to the Ethernet device that harm can venture from.
# such as the internet.
PUBDEV=eth0

# The name of the iptables chain to manage the permanent bans
# within. The name doesn't matter so long that it doesn't
# collide with a chain you're already managing.
# Also, Do not change this value in the future because that
# will hinder the ability to upgrade/append new bans easily.
# It is doubtful that it's current set value will conflict
# with anything. Therefore just leave the name the way it is
# now:
FILTER=nuxref-perm-ban

# The script makes an effort to detect IP Addresses all
# coming from the same Class C type network.  Rather then
# have an entry per IP like fail2ban manages; we group
# them into 1 to speed the look-ups iptables preforms.
# You'll want to identify the minimum number of IP
# addresses matched within the same alike (Class C) network
# before this grouping takes place.
CLASSCGRP_MIN=2

# IP Tables configuration file read during your system
# startup.
IPTABLES_CFG=/etc/sysconfig/iptables

# IPTables Chain Index
# Identify where you want your ban list to be applied
# To reduce overhead, the banlist should be processed 'after'
# some core checks you do.  For example I have a series of other
# checks i do first such as allowing already established
# connections to pass through.  I didn't want the ban list
# being applied to these, so for my system, i set this to 11.
# Setting it to 1 is safe because it guarantees it's at least
# processed first on systems who don't actively maintain their
# own custom firewall list.
CHAINID=1

########################################
# Preparation
########################################

# First we built a massive sorted and unique list of
# already banned IP addresses.
# The below is clever enough to include previous content
# you've banned from before (if any exists):
(
   # carry over old master list
   iptables -L -n | awk "/Chain $FILTER/, $NF ~ /RETURN/" | 
    egrep DROP | sed -e 's/^[^0-9.]*([0-9]+.[0-9]+.[0-9]+).([0-9/]+).*/1 .2/g'
   # update master list with new data
   iptables -L -n | egrep REJECT | 
    sed -e 's/^[^0-9.]*([0-9]+.[0-9]+.[0-9]+).([0-9]+).*/1 .2/g'
) | sort -n | uniq > list
 
# Now we build a separate list (in memory) that will track
# all of the ip addresses that met our $CLASSCGRP_MIN flag.
CLASSC_LIST=$(cat list | cut -f1 -d' ' | uniq -c | 
    sed 's/^ *//g' | sort -n | 
    awk "int($1)>=$CLASSCGRP_MIN" | cut -f2 -d' ')
 
# We eliminate the duplicates already pulled into memory
# from the master list of IPs
for NW in $CLASSC_LIST; do sed -i -e "/^$NW /d" list; done

# Now we scan our list and repopulate them into our master
# list. We place these entries at the head of the file so
# that they'll be added to our iptable ban chain first.
for NW in $CLASSC_LIST; do sed -i "1s/^/$NW .0/24n/" list; done

# Using our list of banned IP addresses, we now generate
# the actual iptable entries (into a file called
# 'commands':
(
   # Creates the chain
   echo iptables -N $FILTER
 
   # Build List of Addresses using our list file
   while read line; do           
      IP=$(echo $line | tr -d '[:space:]')
      echo iptables -A $FILTER -s $IP -j DROP;
   done < list
 
   # Allow future iptable processing to continue 
   # after reading through this chain by appending
   # the RETURN action.
   echo iptables -A $FILTER -j RETURN
 
   # Add chain to INPUT
   echo iptables -t filter -I INPUT $CHAINID -i $PUBDEV -j $FILTER
) > commands

########################################
# IPTables (Temporary Instalment)
########################################

# Have a look at our commands if you want:
cat commands

# Apply these new rules now with the following command:
sh commands

# The commands generated in the 'commands' text file 
# are only temporary; they will be lost if your
# machine (or iptables) is ever restarted

########################################
# IPTables (Permanent Installation)
########################################
# Consider making a backup of your configuration in case you
# need to roll back
/bin/cp -f $IPTABLES_CFG $IPTABLES_CFG.backup

# Now we generate all of the commands needed to place
# into our iptables configuration file:
sed -e 's/^iptables[ ]*//g' commands | 
    egrep "^-A $FILTER -s " > commands-iptables
# Clean up old configuration
sed -i -e "/^:$FILTER -/d" "$IPTABLES_CFG"
sed -i -e "/^-A INPUT .* $FILTER$/d" "$IPTABLES_CFG"
sed -i -e "/^-A $FILTER -/d" "$IPTABLES_CFG"
 
# Now push all of our new ban entries into the iptables file
sed -i -e "/:OUTPUT .*/a:$FILTER - [0:0]" "$IPTABLES_CFG"
sed -i -e "/:$FILTER - .*/a-A INPUT -i $PUBDEV -j $FILTER" "$IPTABLES_CFG"
sed -i -e "/-A INPUT -i $PUBDEV -j $FILTER.*/r commands-iptables" "$IPTABLES_CFG"

# Now preform the following to reset all of the fail2ban
# jails you've got as well load your new permanent ban setup
service fail2ban stop
service iptables restart
service fail2ban start

# If you have a problem; just roll back your backup you
# created and rerun the 3 commands above again. You can
# have a look at the table with the following command:
iptables -L -n -v

########################################
# IPTables (Optional Tiding)
########################################
# the above will insert the banlist at the top
# The below will just correct this and move it
# clean entry(s) from INPUT
(
   while [ 1 -eq 1 ]; do
      ID=$(iptables -nL --line-numbers | 
           egrep "^[0-9]+[ t]+$FILTER " | 
           head -n 1 | 
           sed -e 's/^([0-9]+).*/1/g')
      [ -z "$ID" ] && break;
      iptables -D INPUT $ID
   done
   # Re insert at the correct $CHAINID
   iptables -t filter -I INPUT $CHAINID -i $PUBDEV -j $FILTER
)

# have another look if you want (if you tidied)
iptables -L -n -v

########################################
# Cleanup (Optional)
########################################
# Remove temporary files when your done; or save them if you
# want to port this data to another server:
rm -f list commands commands-iptables

You can undo and destroy the new entries an any time using the following:

# disassociate filter from INPUT
while [ 1 -eq 1 ]; do
   ID=$(iptables -nL --line-numbers | 
        egrep "^[0-9]+[ t]+$FILTER " | 
        head -n 1 | 
        sed -e 's/^([0-9]+).*/1/g')
   [ -z "$ID" ] && break;
   iptables -D INPUT $ID
done
# flush filter
iptables -F $FILTER
# remove filter
iptables -X $FILTER

Credit

This blog took some time to put together and test! If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Sources

There were not many sources used to make this entry. Most of it is just shell scripting knowledge I’ve adopted over the years mixed with some iptable commands. Here are some sources anyway that are applicable:

Linux HOWTO, Linux Tools, Missing Linux Packages

Linux Administration Tools & Tricks

Leave a comment

Introduction

All *nux administrators have tools they use to make their job easier. This blog will be an ongoing one where every time I find or create a handy tool, I’ll share it. I’ll do my best to provide a quick write up with it and some simple examples as to how it may be useful to you too.
Currently, this blog touches on the following:

Applications

Please note that most of these tools can be directly retrieved from my repository I’m hosting. You may wish to connect to it to make your life easier.

genssl: SSL/TLS Certificate Generator and Certificate Validation

genssl is a simple bash script I created to automatically generate Private/Public key pairs that can be used by all sorts of applications requiring SSL/TLS encryption. This could be used by any number of servers you’re hosting (Web, Mail, FTP, etc). It will also test previously created keys (validating them). It supports generating keys that are to be signed by Certificate Authority as well as self signed and test keys too.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref genssl

What You’ll Need To Know
It’s really quite simple, SSL/TLS keys are usually assigned to domain names. Thus this is the only input to the tool in addition to the key type you want to generate (self-signed, test, or registered):

# Generate a simple self-signed Certificate Private/Public
# key pair for the domain example.com you would just type the
# following:
genssl -s example.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README

# You can specify as many domain names as you want to
# generate more then one:
genssl -s example.com nuxref.com myothersite.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README
#   - nuxref.com.key
#   - nuxref.com.crt
#   - nuxref.com.README
#   - myothersite.com.key
#   - myothersite.com.crt
#   - myothersite.com.README

You can verify the keys with the following command:

# The following scans the current directory
# for all .key, .crt, and/or .csr files and tests that
# they all correctly match against one another.
genssl -v

If you need a signed certificate by a registered Certificate Authority, genssl will get you half way there. It will generate you your public key and an unsigned certificate (.csr). You’ll then need to choose a Certificate Authority that works best for you (cost wise, features you need). You will provide them your unsigned certificate you just generated and in return, they’ll produce (and provide) for you the private key you need (.key). Signed (registered) certificates cost money because you’re paying for someone else to confirm the authenticity of your site when people visit it. This is by far the absolute best way to publicly host a secure website!

# Generate an unsigned certificate and public key
# pair for the domain example.com with the following:
genssl -r example.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README

You’ll notice that you also generate a README file for every domain you create keys for. Some of us won’t generate keys often; so this README provides some additional information that will explain how you can use the key. It lists a few Certificate Authoritative locations you can contact, and provides examples on how you can install the key into your system. Depending on the key pair type you generate, the README will provide you with information specific to your cause. Have a look at the contents yourself after you generate a key! I find the information in it useful!

The last thing worth noting about this tool is to create a key pair, you usually provide information about yourself or the site you’re creating the key for. I’ve defaulted all these values to ones you’re probably not interested in using. You can over-ride them by specifying additional switches on the command line. But the absolute easiest (and better) way of doing it, is to just provide a global configuration file it can reference which is /etc/genssl. This file is always sourced (if present). The next file that is checked and sourced is ~/.config/genssl and then ~/.genssl.
Set this with information pertaining to your hosting to ease the generation of of the key details. Here is an example of what any one of these configuration files might look like:

# The country code is represented in it's 2 letter abbreviated version:
# hence: CA, US, UK, etc
# Defaults to "7K" if none is specified
GENSSL_COUNTRY="7K"
# Your organization/company Name
# Defaults to "Lannisters" if none is specified
GENSSL_ORG="Lannisters"
# The Province and or State you reside in
# Defaults to "Westerlands" if none is specified
GENSSL_PROVSTATE="Westerlands"
# Identify the City/Town you reside in
# Defaults to "Casterly Rock" if none is specified
GENSSL_LOCATION="Casterly Rock"
# Define a department; this is loosely used. Some
# just leave it blank
# Defaults to "" (blank) if not is specified
GENSSL_DEPT=""

There are other variables you can override, but the above are the key ones. The default cipher strength is rsa:2048 and keys expire after 730 days (which equates to 2 years). This is usually the max time Certificate Authoritative sites will sign your key for anyway (if registering with them).

You don’t need a configuration file at all for this script to work, there are switches for all of these variables too that you can pass in. Its important to note that passed in switches will always over-ride the configuration file too, but this is the same with most applications anyway.

Some Further Reading

fencemon: A System Monitoring Tool

fencemon is a beautiful creation by Red Hat, the only thing I did was package it as an RPM. I don’t think it’s advertised or used enough by system administrators. It simply runs as a cron and constantly gathers detailed system information. Information such as the current programs running, the system memory remaining, network statistics, disk i/o, etc. It captures content every 10 seconds but does so by using tools that are not i/o intensive. So running this will not slow your machine down enough to justify not using it.

Where this information becomes vital is during a system failure (in the event one ever does occur). It will allow you to see exactly what the last state of the system was (processes in memory, etc) prior to your issue. The detailed information can be used with everyday troubleshooting as well too such as identifying that process that is going crazy overnight and other anonymities that you’re never around to witness first hand.

Why is it called fencemon?
It got it’s name because it’s initial purpose was to run in clustered environments which do a think called ‘fencing’. It’s when another cluster node sees that another is violating some of the simple cluster rules put in place, or it simply isn’t responding anymore. Fencing is effectively the power cycling of the node (so an admin doesn’t have to). In almost all cases (unless there is seriously something wrong with the fenced node), it will reboot and rejoin the cluster correctly. This tool would have allowed an admin to see the final state of the cluster node before it was lost. But any server can crash or go crazy when deploying software in a production environment. We all hope it never happens, but it can. The logging put in place by fencemon can be a life saver!

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref fencemon

What You’ll Need To Know
The constant system capturing that is going on will report itself to /var/log/fencemon/.

The format of the log files is:
hostname–YYYYMMDD–HHMMSS-info-capturetype.log

Once an hour elapses, a new file is written to. Each file contains system statistics in 20 second bursts; as you can imagine, there is A LOT of information here.

the following capturetype log files (with their associated commands) are gathered in 20 second increments:

  • hostname–YYYYMMDD–HHMMSS-info-ps-wchan.log: 
    ps -eo user,pid,%cpu,%mem,vsz,rss,tty,stat,start,time,wchan:32,args
  • hostname–YYYYMMDD–HHMMSS-info-vmstat.log: 
    vmstat
  • hostname–YYYYMMDD–HHMMSS-info-vmstat -d.log: 
    vmstat-d
  • hostname–YYYYMMDD–HHMMSS-info-netstat.log: 
    netstat -s
  • hostname–YYYYMMDD–HHMMSS-info-meminfo.log: 
    cat /proc/meminfo
  • hostname–YYYYMMDD–HHMMSS-info-slabinfo.log: 
    cat /proc/slabinfo
  • hostname–YYYYMMDD–HHMMSS-info-ethtool-$iface.log: 
    # $iface will be whatever is detected on your machine
# you can also define this variable in /etc/sysconfig/fencemon
# ifaces="lo eth0"
# The above definition would cause the following to occur:
/sbin/ethtool -S lo >> 
   /var/log/fencemon/hostname-YYYYMMDD-HHMMSS-info-ethtool-lo.log
/sbin/ethtool -S eth0 >> 
   /var/log/fencemon/hostname-YYYYMMDD-HHMMSS-info-ethtool-eth0.log

The log files are kept for about 2 days which can occupy about 750MB of disk space. But hey, disk space is cheap now of days. You should have no problem reserving a GIG of disk space for this info. It’s really worth it in the long run!

Some Further Reading

datemath: A Date/Time Manipulation Utility

I won’t go too deep on this subject since I’ve already blogged about it before here. In a nutshell, It can easily provide you a way of manipulating time on the fly to ease scripting. This tool is not rocket science by any means, but it simplifies shell scripting a great deal when preforming functionality that is time sensitive.

This is effectively an extension or the the missing features to the already existing date tool which a lot of developers and admins use regularly.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref datemath

What You’ll Need To Know
There are 2 key applications; datemath and dateblock. Datemath can ease scripting by returning you time relative to when it was called; for example:

# Note: the current date/time is printed to the
#        screen to give you an idea how the math was
#        applied.
# date +'%Y-%m-%d %H:%M:%S' prints: 2014-11-14 21:49:42
# What will the time be in 20 hours from now?
datemath -o 20
# 2014-11-15 17:49:42

# Top of the hour 9000 minutes in the future:
datemath -n 9000 -f '%Y-%m-%d %H:00:00'
# 2014-11-21 03:00:00

If you use a negative sign, then it looks in the past. There are lots of reasons why you might want to do this; consider this little bit of code:

# Touch a file that is 30 seconds older than 'now'
touch -t $(datemath -s -30 -f '%Y%m%d%H%M.%S') reference_file

# Now we have a reference point when using the 'find'
# command. The below command will list all files that
# at least 30 seconds old. This might be useful if your
# hosting an FTP server and don't want to process files
# until they've arrived completely.
find -type f -newer reference_file
# You could move the fully delivered files to a new
# directory with a slight adjustment to the above line
find -type f -newer reference_file -exec mv -f {} /new/path ;

# Consider this for an archive clean-up for a system
# you maintain. Simply touch a file as being 31 days
# older then 'now'
touch -t $(datemath -d -31 -f '%Y%m%d%H%M.%S') reference_file

# now you can list all the files that are older then
# 31 days.  Add the -delete switch and you can clean
# up old content from a directory.
find /an/archive/directory -type f ! -newer reference_file

# You can easily script sql statements using this tool too
# consider that you need to just select yesterdays data
# for an archive:
START=$(datemath -d -1 -f '%Y-%m-%d 00:00:00')
FINISH=$(date +'%Y-%m-%d 00:00:00')
BACKUP_FILE=/absolute/path/to/put/backup/$START-backup.csv
# Now your SQL statement could be:
/usr/bin/psql -d postgres 
      -c "COPY (SELECT * FROM mysystem.data WHERE 
                mysystem.start_date >= '$START' AND 
                mysystem.finish_date < '$FINISH') 
          TO '$BACKUP_FILE';"
# Now that we've backed our data up, we can remove
# it from our database:
/usr/bin/psql -d postgres 
      -c "DELETE FROM mysystem.data  WHERE 
                mysystem.start_date >= '$START' AND 
                mysystem.finish_date < '$FINISH'"

Some Further Reading

dateblock: A Cron-Like Utility

dateblock allows us to mimic the functionality of sleep by blocking the process. The catch is dateblock blocks until a specific time instead of for a duration of time.

This very subtle difference can prove to be a very effective and powerful too, especially when you want to execute something at a specific time. Cron’s are fine for most scenarios, but they aren’t ideal in a clustered environment where this tool excels. In a clustered environment you may only want an action to occur on the node hosting the application, where a cron would require the action to fire on all nodes.

The python C++ library enables this tools usage even further since you can integrate it with your application.

Just like datemath is, dateblock is discussed in much more detail in my blog about it here.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref dateblock

# Get the python library too if you want it
yum install -y --enablerepo=nuxref python-dateblock

What You’ll Need To Know
Consider the following:

# block for 30 seconds...
sleep 30

# however dateblock -s 30 would block until the next 30th second
# of the minute is reached. Consider the time:
# date +'%Y-%m-%d %H:%M:%S' prints: 2014-11-14 22:14:16
# dateblock would block until 22:14:30 (just 14 seconds later)
dateblock -s 30

Dateblock works just like a crontab does and supports the crontab format too. It was written by me to specifically allow accuracy for time sensitive applications running. If you write your actions and commands in a script under a dateblock command, you can always guarantee your actions will be called at a precise time.

Since it supports cron entries you can feed it things like this:

# The following will unblock if the minute becomes
# divisible by 10.  so this would unblock at the
# :00, :10, :20, :30: 40: and :50 intervals
dateblock -n /10

# the above command could also be written like this:
dateblock -n 0,10,20,30,40,50

# You can mix and match switches to tweak the tool to
# always work in your favour.

A really nice switch that can help with your debugging and development is the –test (-t) switch. This will cause dateblock to ‘not’ block at all. Instead it will just spit out the current time, followed by the time it ‘would’ have unblocked at had you not put it into test mode. This is good for trying to tweak complicated arguments to get the tool to work in your favour.

# using the test switch, we can make sure we're going
# to unblock at time intervals we expect it to. In this
# example, i use the test switch and a cron formatted
# argument.  In this example, I'm asking it to unblock
# ever 2 days at the start of each day (midnight with
# respect to the systems time)
dateblock -t -c "0 0 0 /2"
Current Time : 2014-11-14 22:24:06 (Fri)
Block Until  : 2014-11-16 00:00:00 (Sun)

# Note: there is an extra 0 above that may through you
#       off.. in the normal cron world, the first
#       argument is the 'minute' entry.  Well since
#       dateblock offers high resolution, the first
#       entry is actually 'seconds', then minute, etc.

There are man pages for both of these tools. You’ll get a lot more detail of the power they offer. Alternatively, if you even remotely interested, check out e my blog entry on them.

The other cool thing is dateblock also has a python library I created.

# Import the module
from dateblock import dateblock

# Here is a similar example as above and we set
# block to false so we know how long were blocking until
unblock_dt = dateblock('0 0 0 /2', block=False)
print("Blocking until %s" % unblock_dt
              .strftime('%Y-%m-%d %H:%M:%S))
# by default, blocking is enabled, so now we can just
# block for the specified time
datetime.datetime('0 0 0 /2')

Some Further Reading

nmap (Network Mapper)

nmap is port scanner & network mapping tool. You can use it to find out all of the open ports a system has on it and you can also use it to see who is on your network. This tool is awesome, but it’s intentions can be easily abused. It’s common sense to scan your own network to make sure that only the essential ports are open to the outside world, but it’s not kind to scan a system to which you are not the owner of. Hence, this tool should be used to identify your systems weaknesses so that you can fix them; it should not be used to identify an others weakness. For this reason, if you do install nmap on your system, consider limiting permission so that only the root user has access to it.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref-shared nmap

# Restrict its access (optional, but ideal)
chmod o-rwx /usr/bin/nmap

What You’ll Need To Know
Now you can do things like:

  • Scan your system to identify all open ports: 
    # Assuming the server you are scanning is 192.168.1.10
nmap 192.168.1.10
  • Scan an entire network to reveal everyone on it (including MAC Address information): 
    # Assuming a class C network (24 masked bits) with
# a network address of 192.168.1.0
nmap -n -v -sP 192.168.1.0/24

Note: Don’t panic if your system appears to have more ports open then you had thought. Well at least don’t panic until you determine where you are running your network map test from. You see, if you run nmap on the same system you are testing against, then there your test might not prove accurate since firewalls do not normally deny any local traffic. The best test is to use a second access point to test against the first. Also, if your server sits on more then one network, make sure to test it from a different server on each network it connects to!

Some Further Reading

  • A blog with a ton of different examples identifying things you can do with this tool.

lsof (LiSt Open Files)

This tool is absolutely fantastic when it comes to preforming diagnostics or handling emergency scenarios. This command is only available to the root user; this is intentional due to the power it offers the system administrators.

I’m not hosting this tool simply because it ships with most Linux flavors (Fedora, CentOS, Red Hat, etc). Therefore, the following should haul it into your system if it’s not already installed:

# just type the following:
yum install -y lsof

# If this doesn't work, the rpm will be found
# on your DVD/CD containing your Linux Operating
# System.

What You’ll Need To Know
This tool when executed on it’s own (without any options) simply lists every open file, device, socket you have on one great big list. This can be useful when paired with grep if your looking for something specific.

# list everything opened
lsof

But it’s true power comes from it’s ability to filter through this list and fetch specific things for you.

# list all of the processes utilizing port 80 on your system:
lsof -i :80 -t

# or be more specific and list all of the processes accessing
# a specific service you are hosting on an IP listening on
# port 80:
# Note: the below assumes we're interested in those accessing
#       the ip address 192.168.0.1
lsof -i @192.168.0.1:80 -t

# If you pair this with the kill command, you can easily kick
# everyone off your server this way:
kill -9 $(lsof -i @192.168.0.1:80 -t)

But the tool is also great for even local administration; let’s say there is a user account on your system with the login id of bob.

# find out what bob is up to and what he's accessing on
# your system this way (you may need to pair this with
# grep since the output can be quite detailed
lsof -u bob

# Perhaps you just want to see all of the network activity
# bob is involved in. The following spits out a really
# easy to read list of all of the network related processes
# bob is dealing with.
lsof -a -u bob -i

# You can kill all of the TCP/IP processes bob is using with
# the following (if bob was violating the system, this might
# be a good course of action to take):
kill -9 $(lsof -t -u bob)

Perhaps you have a file on your system you use regularly, you can find out who is also using it with the following command:

# List who is accessing a file
lsof /path/to/a/file

Some Further Reading

  • lsof Survival Guide: A Stack Overflow post with some awesome tricks you can do with this tool.
  • More great lsof examples. This is someones blog who specifically wrote about this tool specifically. They provide many more documented examples of this tools usage here.

Credit

This blog took me a very long time to put together and test! If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Linux HOWTO

Configuring a DNS Server on CentOS 6

Leave a comment

Introduction

We have been relying on the Domain Name System (DNS) since the dawn of the internet. Simply put: it allows us to access information by a human readable string or recognizable name such as google.com or nuxref.com instead of it’s actual IP address (which is not as easily memorizable). If we didn’t have the DNS, then the internet would not have evolved as far as it has today. The DNS was built on a series of Name Servers that are all looking after their respected domain (or zone). Our Internet Service Provider (ISP) is lending us their DNS servers everyday when we connect to them. It’s our wireless router (at home or at work) that passes this server to our tablet, phone, laptop etc… when we connect to it.

Here is a simple DNS query taking place illustrating how most of us are setup today.
Here is a simple DNS query taking place illustrating how most of us are setup today.
Managing our own Authoritative DNS Server allows us to catalog our personal devices we use daily with great ease. If you’re publicly hosting content, an Authoritative DNS server can be used to even distribute the traffic you servers receive both geographically and as a distributed (load balancing) approach. It gives us the ability to dynamically associate names to all of our devices on our network. It’s great for the hobbyist and absolutely mandatory for any medium or larger sized company.

PowerDNS is my preferred DNS server solution. I personally prefer this to it’s long-term predecessor Berkeley Internet Name Domain (BIND). BIND has been around since 1984 and has gone through years of hacky patches to get to where it is today. PowerDNS is much younger (first release was in 1999), but was written without all of the growing pains BIND suffered through from the start. In all fairness, BIND developers were forced to deal with RFC (Request for Comments) as DNS continued to evolve to what it is today. Where as PowerDNS already had a stable set of requirements to work with from day one. Not to mention PowerDNS can be easily configured to use alternative backend databases.

You are reading this blog because you want the following:

  • A fast and reliable Authoritative DNS server with a PostgreSQL database backend.
  • You want a central configuration point; you want everything to be easy to maintain after it’s all set up.
  • You want everything to just work the first time and you want to leave the figuring it out part to the end.
  • Package management and version control is incredibly important to you.
  • You want the ability to catalog your local network by assigning devices on it their own unique (easy to remember) hostnames.
  • You want to maintain the ability to surf the internet by forwarding on requests your DNS server doesn’t know to another that does.
The beauty of running your own Authorative DNS grants you the ability to catalog and easily access everything on your local network.
The beauty of running your own Authorative DNS grants you the ability to catalog and easily access everything on your local network by it’s hostname (you assign).

Here is what my tutorial will be focused on:

  • PowerDNS (v3.x) configured to use a Database Backend (PostgreSQL) giving you central configuration. This tutorial focuses on version 8.4 because that is what ships with CentOS and Red Hat. But most (if not all) of this tutorial should still work fine if you choose to use version 9.x of the database instead.
  • PowerDNS Recursor (v3.x) will be configured to handle anything records we don’t otherwise host or override
  • Security Considered
  • Poweradmin (v2.x) will provide our administration of the DNS records we add via it’s simple web interface.

Please note the application versions identified above as this tutorial focuses specifically on only them. One big issue I found while researching how to set up thing on the net was some tutorials I found didn’t really mention the version they were using. Hence, when I would stumble across these old article(s) with new(er) software, it would make for quite a painful experience when things didn’t work.

Please also note that other tutorials will imply that you setup one feature at a time. Then test it to see if it worked correctly before moving on to the next step. This is no doubt the proper way to do things. However, I’m just going to give it all to you at once. If you stick with the versions and packages I provide… If you follow my instructions, it will just work for you the first time. Debugging on your end will be a matter of tracing back to see what step you missed.

I tried to make this tutorial as cookie cutter(ish) as I could. Therefore you can literally just copy and paste what I share right to your shell prompt and the entire setup will be automated for you.

Installation

The following four (4) steps will get you set up with your very own DNS server.

Step 1 of 4: Setup Your Environment

This is the key to my entire blog; it’s going to make all of the remaining steps just work the first time for you. All; I repeat All of the steps below (after this one) assume that you’ve set this environment up. You will need to reset up your environment at least once before running through any of the remaining steps below or they will not work.

It’s also important to mention that you will need to be root to configure the DNS server. This applies to all of the steps identified below throughout this blog.

I re-hosted all of the packages I used to successfully pull this blog off. This allows me to host this information and pair it with the software it works against. Feel free to hook up to my repositories to speed up your setup.

Install all of the necessary packages:

# Connect to my repository to which I've had to rebuild a few
# packages to support PostgreSQL as well as fix some bugs in
# other bugs. This step will really make your life easy and let
# us compare apples to apples with package versions. It also
# allows you to haul in a working setup right out of the box.
#
# Be sure you're connected to my repository for the below to work
# visit: http://nuxref.com/nuxref-repository/

################################################################
# Install our required products
################################################################
yum install -y 
       --enablerepo=nuxref 
       --enablerepo=nuxref-shared 
           postgresql-server postgresql 
           php-pgsql php-imap php-mcrypt php-mbstring  
           pdns pdns-backend-postgresql pdns-recursor 
           poweradmin 
           nuxref-templates-pdns

# Also make sure these products are installed as well since we
# use them to manipulate and test some of the data
yum install -y awk sed bind-utils curl

# Choose between NginX or Apache
## NginX Option (a) - This one is my preferred choice:
yum install -y 
       --enablerepo=nuxref 
       --enablerepo=nuxref-shared 
           nginx php-fpm

## Apache Option (b):
yum install -y 
       --enablerepo=nuxref 
       --enablerepo=nuxref-shared 
            httpd php

# Setup Default Timezone for PHP. For a list of supported
# timezones you can visit here: http://ca1.php.net/timezones
TIMEZONE="America/Montreal"
sed -i -e "s|^[ t]*;*(date.timezone[ t]*=).*|1 $TIMEZONE|g" 
    /etc/php.ini

# Ensure we're not using Strict PHP Handling
sed -i -e 's/^[ t]*(error_reporting)[ t]*=.*$/1 = E_ALL & ~E_STRICT/g' 
    /etc/php.ini 

################################################################
# Setup PostgreSQL (v8.4)
################################################################
# The commands below should all work fine on a PostgreSQL v9.x
# database too; but your mileage may vary as I've not personally
# tested it yet. You can skip this section if you've already
# got a database running using one of my earlier tutorials.

# Only init the database if you haven't already. This command
# could otherwise reset things and you'll loose everything.
# If your database is already setup and running, then you can
# skip this line
service postgresql initdb

# Now that the database is initialized, configure it to trust
# connections from 'this' server (localhost)
sed -i -e 's/^[ t]*(local|host)([ t]+.*)/#12/g' 
    /var/lib/pgsql/data/pg_hba.conf
cat << _EOF >> /var/lib/pgsql/data/pg_hba.conf
# Configure all local database access with trust permissions
local   all         all                               trust
host    all         all         127.0.0.1/32          trust
host    all         all         ::1/128               trust
_EOF

# Make sure PostgreSQL is configured to start up each time
# you start up your system
chkconfig --levels 345 postgresql on

# Start the database now too because we're going to need it
# very shortly in this tutorial
service postgresql start

To simplify your life, I’ve made the configuration of all the steps below reference a few global variables. The ones identified below are the only ones you’ll probably want to change. May I suggest you paste the below information in your favourite text editor (vi, emacs, etc) and adjust the variables to how you want them making it easier to paste them back to your terminal screen.

# The following is only used for our SSL Key Generation.
# You can skip SSL Key generation if you've done so using an
# earlier tutorial
COUNTRY_CODE="7K"
PROV_STATE="Westerlands"
CITY="Lannisport"
SITE_NAME="NuxRef"

Now for the rest of the global configuration; There really should be no reason to change any of these values (but feel free to). It’s important that you paste the above information (tailored to your liking’s) as well as the below information below to your command line interface (CLI) of the server you wish to set up.

# PostgreSQL Database
PGHOST=localhost
PGPORT=5432
PGNAME=system_dns
PGRWUSER=pdns
PGRWPASS=pdns

# Identify the domain name of your server here
# I use the .local extension because I only intend to resolve
# internal addresses with my DNS server.  You may wish to use
# a different value.
DOMAIN=nuxref.local

# Configure a recursor, the recursor will cache your database hits
# and will greatly increase performance.  Ideally you want to set
# the recursor address to the address you want to host your server
# on.  This is the same IP address you will add to everyones 
# /etc/resolve.conf later.  This is in fact your name server.
# If you leave this value at 127.0.0.1 your DNS will be restricted
# to just the server you're hosting on.
# If you aren't sure what your IP Address is, you can just type 'ifconfig'
#
# This command may also fetch your ip address:
# cat /etc/sysconfig/network-scripts/ifcfg-* | 
#    egrep '^IPADDR=' | egrep -v '127.0.0.1' | 
#    cut -f2 -d'=' | head -n1
NAMESERVER_ADDR=$(cat /etc/sysconfig/network-scripts/ifcfg-* | 
    egrep '^IPADDR=' | egrep -v '127.0.0.1' | 
    cut -f2 -d'=' | head -n1)
# Alternatively, if you're not reading this and 'only' if the
# above failed we'll just set the address to your local address.
NAMESERVER_ADDR=${NAMESERVER_ADDR:=127.0.0.1}

# The Network that our DNS server resides on is important information
# for security purposes.  We want to only allow recursion on this
# network alone and not others hitting our server.  The below
# looks kind of cryptic, but it's just a method of extracting
# the network information automatically if you don't already
# know it. It may or may not work; it will depend on if you set
# a proper NAMESERVER_ADDR
SUBNET_ADDR=$(/sbin/ifconfig | egrep -m1 $NAMESERVER_ADDR | 
    sed -e 's/.*Mask:([0-9.]+).*/1/g')
NAMESERVER_PRFX=$(ipcalc -s -p $NAMESERVER_ADDR $SUBNET_ADDR | 
                   cut -f2 -d'=')
# Assign a default in case the above command failed.
NAMESERVER_PRFX=${NAMESERVER_PRFX:=24}

# Calculate our network
NAMESERVER_NWRK=$(ipcalc -s -n $NAMESERVER_ADDR $SUBNET_ADDR | 
                   cut -f2 -d'=')
# Assign a default in case the above command failed.
NAMESERVER_NWRK=${NAMESERVER_NWRK:=$(echo $NAMESERVER_ADDR | 
                   cut -f1,2,3 -d'.').0}

# Reverse Address Resolution Preparation
# This converts and IP Address of 1.2.3.4 to 3.2.1.in-addr.arpa
# We can use this later to create a reverse translation which
# PowerDNS can administrate for us also.  The templates I created
# will set some early examples up for you.
NAMESERVER_ARPA=$(echo "$NAMESERVER_ADDR" | 
    awk -F"." '{print $3 "." $2 "." $1 ".in-addr.arpa"}')

# We now need the 4th octet of our Name Server Address to complete
# our ARPA address for the reverse lookup. For example, if your server
# ip is 2.4.8.16, we want the '16' defined here.  The below is just
# a cheat to go ahead and extract it from the address you specified
NAMESERVER_OCT4=$(echo "$NAMESERVER_ADDR" | 
    cut -f4 -d'.')

# This is where our templates get installed to make your life
# incredibly easy and the setup to be painless. These files are
# installed from the nuxref-templates-pdns RPM package you
# installed above. If you do not have this RPM package then you
# must install it or this blog simply won't work for you.
# > yum install --enablerepo=nuxref nuxref-templates-pdns
NUXREF_TEMPLATES=/usr/share/nuxref

I realize the above environment can seem a bit cryptic. I tried to simplify this DNS setup so that even a novice’s life would be easy. The environment variables attempt to detect everyones settings automatically. In some cases, I may have just made it worse for some (hopefully not). It would be a good idea to just echo the defined variables to your screen and confirm they are as you expect them to be. They really are the key to making all of the next steps work in this blog.

# Simple Check
# Note: grab the brackets too when you copy and paste the below
(
   for VAR in COUNTRY_CODE PROV_STATE CITY SITE_NAME 
              PGHOST PGPORT PGNAME PGRWUSER PGRWPASS 
              DOMAIN NAMESERVER_ADDR NAMESERVER_PRFX 
              NAMESERVER_NWRK NAMESERVER_ARPA 
              NAMESERVER_OCT4 NUXREF_TEMPLATES; do
      [ -z $(eval "echo $$VAR") ] && echo "You must set the variable: $VAR"
   done
)
# Pretty Printing
# Note: grab the brackets too when you copy and paste the below
(
   echo "PostgreSQL:"
   echo -e "tPGHOST=$PGHOSTntPGPORT=$PGPORTntPGNAME=$PGNAMEntPGRWUSER=$PGRWUSERntPGRWPASS=$PGRWPASSn"
   echo "SSL:"
   echo -e "tCOUNTRY_CODE='$COUNTRY_CODE'ntPROV_STATE='$PROV_STATE'ntCITY='$CITY'ntSITE_NAME='$SITE_NAME'n"
   echo "Nameserver:"
   echo -e "tDOMAIN=$DOMAINntNAMESERVER_ADDR=$NAMESERVER_ADDRntNAMESERVER_NWRK=$NAMESERVER_NWRKntNAMESERVER_PRFX=$NAMESERVER_PRFX"
   echo -e "tNAMESERVER_ARPA=$NAMESERVER_ARPAntNAMESERVER_OCT4=$NAMESERVER_OCT4n"
   echo "NuxRef Templating"
   echo -e "tNUXREF_TEMPLATES=$NUXREF_TEMPLATES"
   echo
)

Step 2 of 4: Setup PowerDNS

First off, make sure you’ve set up your environment correctly (defined in Step 1 above) or you will have problems with the outcome of this step!
Database Configuration:

################################################################
# Configure PostgreSQL (for PowerDNS)
################################################################
# Optionally Eliminate Reset Database.
/bin/su -c "/usr/bin/dropdb -h $PGHOST -p $PGPORT $PGNAME 2>&1" postgres &>/dev/null
/bin/su -c "/usr/bin/dropuser -h $PGHOST -p $PGPORT $PGRWUSER 2>&1" postgres &>/dev/null

# Create Read/Write User (our Administrator)
echo "Enter the role password of '$PGRWPASS' when prompted"
/bin/su -c "/usr/bin/createuser -h $PGHOST -p $PGPORT -S -D -R $PGRWUSER -P 2>&1" postgres

# Create our Database and assign it our Administrator as it's owner
/bin/su -c "/usr/bin/createdb -h $PGHOST -p $PGPORT -O $PGRWUSER $PGNAME 2>&1" postgres 2>&1

# the below seems big; but will work fine if you just copy and
# it as is right to your terminal: This will prepare the SQL
# statement needed to build your DNS server's database backend
sed -e '/^--?/d' 
    -e "s/%PGRWUSER%/$PGRWUSER/g" 
        $NUXREF_TEMPLATES/pgsql.pdns.template.schema.sql > 
          /tmp/pgsql.pdns.schema.sql

# load DB
/bin/su -c "/usr/bin/psql -h $PGHOST -p $PGPORT -f /tmp/pgsql.pdns.schema.sql $PGNAME 2>&1" postgres 2>&1
# cleanup
/bin/rm -f /tmp/pgsql.pdns.schema.sql

# This will get your database started with some working data to use.
# This part is optional, but since it's so easy to delete stuff later
# and there really isn't a whole lot taking place here, you should run
# this step. It becomes especially useful in debugging later.
sed -e "/^--?/d" 
    -e "s/%DOMAIN%/$DOMAIN/g" 
    -e "s/%NAMESERVER_ADDR%/$NAMESERVER_ADDR/g" 
    -e "s/%NAMESERVER_ARPA%/$NAMESERVER_ARPA/g" 
    -e "s/%NAMESERVER_OCT4%/$NAMESERVER_OCT4/g" 
        $NUXREF_TEMPLATES/pgsql.pdns.template.data.sql > 
            /tmp/pgsql.pdns.data.sql

# load DB with our data
/bin/su -c "/usr/bin/psql -h $PGHOST -p $PGPORT -f /tmp/pgsql.pdns.data.sql $PGNAME 2>&1" postgres 2>&1
# cleanup
/bin/rm -f /tmp/pgsql.pdns.data.sql

Server Configuration:

################################################################
# Configure PowerDNS
################################################################
# Create backup of configuration files
[ ! -f /etc/pdns/pdns.conf.orig ] && 
   cp /etc/pdns/pdns.conf /etc/pdns/pdns.conf.orig

# Install our configuration using the template
sed -e "/^#?/d" 
    -e "s/%NAMESERVER_ADDR%/$NAMESERVER_ADDR/g" 
    -e "s/%NAMESERVER_NWRK%/$NAMESERVER_NWRK/g" 
    -e "s/%NAMESERVER_PRFX%/$NAMESERVER_PRFX/g" 
    -e "s/%PGRWUSER%/$PGRWUSER/g" 
    -e "s/%PGRWPASS%/$PGRWPASS/g" 
    -e "s/%PGHOST%/$PGHOST/g" 
    -e "s/%PGPORT%/$PGPORT/g" 
    -e "s/%PGNAME%/$PGNAME/g" 
        $NUXREF_TEMPLATES/pgsql.pdns.template.pdns.conf > 
          /etc/pdns/pdns.conf

# Protect our configuration since it has user/pass info
# inside of it.
chmod 640 /etc/pdns/pdns.conf
chown root.pdns /etc/pdns/pdns.conf

Step 3 of 4: Setup PowerDNS Recursor

################################################################
# Configure PowerDNS Recursor
################################################################
# Create backup of configuration files
[ ! -f /etc/pdns-recursor/recursor.conf.orig ] && 
   cp /etc/pdns-recursor/recursor.conf 
        /etc/pdns-recursor/recursor.conf.orig

# Install our configuration using the template
sed -e "/^#?/d" 
        $NUXREF_TEMPLATES/pgsql.pdns-recursor.template.recursor.conf > 
          /etc/pdns-recursor/recursor.conf

# Generate an up to date root.hints file, this allows recursion
# back out to the internet.
curl -u ftp:ftp 'ftp://ftp.rs.internic.net/domain/db.cache' 
    -o /etc/pdns-recursor/root.hints

# If the above command did not work, you can use the one I shipped
# with the nuxref-template.pdns packaging:
#   cp $NUXREF_TEMPLATES/root.hints /etc/pdns-recursor/root.hints

# Alternatively, PowerDNS is hardcoded with a default set of root-hints.
# But i personally just like seeing it as an external configuration instead
# But... if all of this is combersom to you and you simply don't want
# to use the offical root.hints and the hard-coded one instead you can
# do the following:
#
# sed -i -e '/^([ t]*hint-file=.*)/d' /etc/pdns-recursor/recursor.conf

# Start up all of our services
chkconfig pdns-recursor --level 345 on
chkconfig pdns --level 345 on
service pdns-recursor restart
service pdns restart

It’s important to take a time-out on this step just to make sure everything is working.
A few simple commands should work perfectly for you otherwise we have an issue:

# The following command should output a bunch of googles DNS servers
nslookup google.com $NAMESERVER_ADDR
# The following command should output the same list
nslookup -port=5300 google.com 127.0.0.1

# If you receive an error such as 
#      ** server can't find google.com: NXDOMAIN
# Then you need to revisit the above steps again

# Alternatively, if you receive an error such as:
#  ;; connection timed out; trying next origin
#  ;; connection timed out; trying next origin
#  ;; connection timed out; no servers could be reached
# Then you have been most likely been restricted access
# to port 53 to the outside world. You're not really
# in a problem state at this point. Make sure the rest
# of the tests (Below) work and then make sure to follow
# the section of this blog entitled:
#     'Zone Forwarding Alternative'
# 
# 
# You should be able to resolve the domain
# poweradmin.$DOMAIN to this very server your hosting
# on:
nslookup poweradmin.$DOMAIN $NAMESERVER_ADDR

# You can even test reverse lookups using our data
# we loaded with the following command:
nslookup $NAMESERVER_ADDR $NAMESERVER_ADDR

# The above should resolve itself to hostmaster.your.domain

Step 4 of 4: Setup PowerAdmin

First off, make sure you’ve set up your environment correctly (defined in Step 1 above) or you will have problems with the outcome of this step!

################################################################
# Configure PostgreSQL (for PowerAdmin)
################################################################

# Now we need to update our database with a schema for
# poweradmin to work with
sed -e "/^--?/d" 
    -e "s/%DOMAIN%/$DOMAIN/g" 
    -e "s/%PGRWUSER%/$PGRWUSER/g" 
        $NUXREF_TEMPLATES/pgsql.poweradmin.template.schema.sql > 
            /tmp/pgsql.poweradmin.schema.sql

# Now we can load the file:
/bin/su -c "/usr/bin/psql -h $PGHOST -p $PGPORT -f /tmp/pgsql.poweradmin.schema.sql $PGNAME 2>&1" postgres 2>&1
# cleanup
/bin/rm -f /tmp/pgsql.poweradmin.schema.sql

# If you loaded the sample dataset for PowerDNS earlier, then you'll
# want to additionally load this file too to help PowerAdmin access it
sed -e "/^--?/d" 
    -e "s/%DOMAIN%/$DOMAIN/g" 
    -e "s/%NAMESERVER_ADDR%/$NAMESERVER_ADDR/g" 
    -e "s/%NAMESERVER_ARPA%/$NAMESERVER_ARPA/g" 
        $NUXREF_TEMPLATES/pgsql.poweradmin.template.data.sql > 
            /tmp/pgsql.poweradmin.data.sql

# load DB with our data
/bin/su -c "/usr/bin/psql -h $PGHOST -p $PGPORT -f /tmp/pgsql.poweradmin.data.sql $PGNAME 2>&1" postgres 2>&1
# cleanup
/bin/rm -f /tmp/pgsql.poweradmin.data.sql

################################################################
# Configure PowerAdmin (for PowerDNS Administration)
################################################################
# Create backup of configuration files
[ ! -f /etc/poweradmin/config.inc.php.orig ] && 
   cp /etc/poweradmin/config.inc.php 
        /etc/poweradmin/config.inc.php.orig

# Apply our configuration
sed -e "/^//?/d" 
    -e "s/%DOMAIN%/$DOMAIN/g" 
    -e "s/%PGHOST%/$PGHOST/g" 
    -e "s/%PGNAME%/$PGNAME/g" 
    -e "s/%PGPORT%/$PGPORT/g" 
    -e "s/%PGRWUSER%/$PGRWUSER/g" 
    -e "s/%PGRWPASS%/$PGRWPASS/g" 
        $NUXREF_TEMPLATES/pgsql.poweradmin.template.config.inc.php > 
            /etc/poweradmin/config.inc.php

# Protect file since it contains passwords
chmod 640 /etc/poweradmin/config.inc.php
chown root.apache /etc/poweradmin/config.inc.php

# NginX Configuration
sed -e "/^#?/d" 
    -e "s/%DOMAIN%/$DOMAIN/g" 
        $NUXREF_TEMPLATES/nginx.poweradmin.template.conf > 
            /etc/nginx/conf.d/poweradmin.conf

################################################################
# Generate SSL Keys For Webpage Security
################################################################
# Generate SSL Keys (if you don't have any already) that we
# will secure all our inbound and outbound mail as.
openssl req -nodes -new -x509 -days 730 -sha256 -newkey rsa:2048 
   -keyout /etc/pki/tls/private/$DOMAIN.key 
   -out /etc/pki/tls/certs/$DOMAIN.crt 
   -subj "/C=$COUNTRY_CODE/ST=$PROV_STATE/L=$CITY/O=$SITE_NAME/OU=IT/CN=$DOMAIN"

# Permissions; protect our Private Key
chmod 400 /etc/pki/tls/private/$DOMAIN.key

# Permissions; protect our Public Key
chmod 444 /etc/pki/tls/certs/$DOMAIN.crt

At this point you should be able to start NginX. If it’s already running
send it a reload or just run the below command.

# If you chose the NginX approach you'll want to make sure it's
# setup to run correctly and restart itself if the system is
# ever restarted:

# Ensure NginX runs even after a reboot
chkconfig nginx --level 345 on
chkconfig php-fpm --level 345 on

# Restart the service if it isn't running already
service php-fpm restart
service nginx restart

Now, we’re almost done. We need to make sure our server is referencing our new DNS server. You may need to update your network settings, but the following will just cheat for the time being and set you up:

[ ! -f /etc/resolv.conf.orig ] && 
   cp /etc/resolv.conf 
       /etc/resolv.conf.orig

# Tell our server to use our new DNS server
cat << _EOF > /etc/resolv.conf
search $DOMAIN
nameserver $NAMESERVER_ADDR
_EOF

# Restore your old configuration like so
# if you need to:
#  /bin/mv -f /etc/resolv.conf.orig /etc/resolv.conf

You will want to additionally add the following to your iptables /etc/sysconfig/iptables:

#---------------------------------------------------------------
# DNS Traffic
#---------------------------------------------------------------
-A INPUT -m state --state NEW -m tcp -p tcp --dport 53 -j ACCEPT
-A INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT

#---------------------------------------------------------------
# Web Traffic for PowerAdmin
#---------------------------------------------------------------
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT

Use the login/pass as admin/admin when you log in for the first time. Consider changing this afterwards!
Use the login/pass as admin/admin when you log in for the first time. Consider changing this afterwards!
You should now be able to visit https://poweradmin and see a login screen. You may have to accept the ‘untrusted key’ prompt. Don’t worry; it’s safe to do so! In fact, if you’re worried, then just have a look at the key itself before accepting it. You’ll see that it’s just the one we generated earlier. The login is admin and the password is admin at the start. You will want to consider changing this right away after you log in for percautionary sake.

Your setup can now be illustrated by the model below. It’s virtually the same setup as you had before however now instead of querying your ISPs DNS Server, you query your very own local one. Now you can easily maintain your own local network and begin labeling the devices you use on it using PowerAdmin.

This illustration shows the PowerDNS Recursor (pdns-recursor)
This illustration shows the PowerDNS Recursor (pdns-recursor)
Your Authorative (Power)DNS Server caches the location for you for a period of time making subsiquent requests to the same spot VERY fast.
All Subsequent Requests are Cached for a Period of Time.
All Subsequent Requests are Cached for a Period of Time.

Zone Forwarding Alternative

Up until now, our ISP was using it’s own root.hints file (or some alternative method) to look up your request. But now, it is our server that is going out directly into the big bad internet instead using this technique. Since DNS requests are not encrypted, it’s now possible for others to spy on the hostnames we’re resolving (and places we’re are visiting). Not only that, these same people can easily trace the source back to us (all DNS requests originate from our IP now). This can allow someone to specifically know the online banking site you use as an example. Prior to hosting your own DNS server, all websites and servers you accessed were channeled privately between you and your ISP so this was never a problem. It was our ISP who made the (recursive) requests for us instead of us doing them ourselves. Prior to now, what we looked up didn’t explicitly trace back to you, it traced back to our ISP. Previously, we actually had more privacy (depending on the contract we signed with our ISP).

Your ISP has thousands of clients making requests to it’s DNS servers constantly. As a result, it has probably already cached 90% of all the websites we intend to visit. Cached content means a very a speedy response from our server. Meanwhile, our local DNS server’s cache will (probably) be empty most of the time (depending on how many people will use it). Hence your ISP’s DNS Server will be MUCH faster then yours.

When you signed up with your ISP, they would have gave you (at least) 1 DNS server to use (most provide 2 – a primary and backup). We can actually tell our DNS Server to use these instead of our root.hints file when it finds a domain that needs to be further looked up. This way, you regain your secure pipe between you and your ISP. The trade off is your adding one more hop to your recursive lookups. But in most scenarios, they would have already cached what your looking for, so it would be an imediate response. The below diagram illustrates the worst case scenario:

A forwarding zone of '*' (asterix) tells the PowerDNS Recursor to forward all requests to a specific Server. In our example we use our ISP's DNS Servers.
A forwarding zone of ‘*’ (asterix) tells the PowerDNS Recursor to forward all requests to a specific Server. In our example we use our ISP’s DNS Servers.

Here is how you can alter your configuration:

# Put your DNS Servers below,  the ones in place right
# now are the public ones offered by Google.
DNS_SERVERS="8.8.8.8 8.8.4.4"

# Remove any information that may conflict
sed -i -e '/^([ t]*hint-file=.*)/d' /etc/pdns-recursor/recursor.conf
sed -i -e '/^([ t]*forward-zones=.*)/d' /etc/pdns-recursor/recursor.conf

# Disable hint-file
echo 'hint-file=' >> /etc/pdns-recursor/recursor.conf

# Prepare Forwarding Zones for everything unmatched:
echo -n 'forward-zones=*=' >> /etc/pdns-recursor/recursor.conf
echo $(echo "$DNS_SERVERS" | 
    sed -e 's/^[ t]*//g' -e 's/[ t]*$//g' -e 's/[ t]+/, /g') >> 
     /etc/pdns-recursor/recursor.conf

# Now restart our recursor
service pdns-recursor restart

Got Old BIND Configuration You Need Imported?

This step is completely optional! If your not familiar with what BIND even is, or know you’ve never used it, you can freely skip this section.
If you migrating from BIND to PowerDNS then you may have a setup in place. PowerDNS makes an easy transition by writing a tool that will scan your old BIND configuration and generate the SQL needed for an easy migration to PowerDNS.

################################################################
# Generate SQL content from all of your zone files
################################################################
# I just had 1 simple DNS zone, but you may many.
# The below did all the work for me (bind was configured to
# run in chroot environment):
# zone2sql --gpgsql --zone=/var/chroot/var/named/data/zone.nuxref.local > 
#    /tmp/pgsql.pdns.zones.sql
# zone2sql --gpgsql --zone=/var/chroot/var/named/data/192.168.0 >> 
#     /tmp/pgsql.pdns.zones.sql

# You could even cheat and run all your files with a command like this
# Please note that this is optional (and not part of the blog, it's just
# a simple conversion tool for those who already have bind configuration
ZONE_DIR=/var/chroot/var/named/data/
[ -f /tmp/pgsql.pdns.zones.sql ] && /bin/rm -f /tmp/pgsql.pdns.zones.sql
for ZONE in $(find $ZONE_DIR -type f); do
   # Fetch ORIGIN/ZONE ID
   ZONE_ID=$(cat $ZONE | egrep '^[ t]*$ORIGIN' | 
              sed -e 's/^.*$ORIGIN[ t]+([^ t]+).*/1/g' 
                  -e 's/[. t]*$//g')
   [ -z "$ZONE_ID" ] && echo "Error Parsing: $ZONE" && continue
   zone2sql --gpgsql --zone=$ZONE --zone-name=$ZONE_ID >> /tmp/pgsql.pdns.zones.sql
done

# Now before you load this file into your database, you may
# want to review it.  It doesn't hurt to scan it over and remove
# any entries you don't think would be useful.

# Under normal circumstances you would be done at this point, however because
# we are additionally using poweradmin, we need to create a few zone entries
# based on the SQL file we just generated.
$NUXREF_TEMPLATES/import2zone.awk /tmp/pgsql.pdns.zones.sql >> 
    /tmp/pgsql.pdns.zones.sql

# Then I just loaded the file straight into the database:
/bin/su -c "/usr/bin/psql -h $PGHOST -p $PGPORT -f /tmp/pgsql.pdns.zones.sql $PGNAME 2>&1" postgres 2>&1
# cleanup
/bin/rm -f /tmp/pgsql.pdns.zones.sql
# You're done!

So… That’s it? Now I’m done?

Yes and No… My blog pretty much hands over a working DNS server with little to no extra configuration needed on your part.

No system is bulletproof; disaster can always strike when you’re least expecting it. To cover yourself, always consider backups of the following:

  • Your PostgreSQL Database: This is where all of your DNS configuration is stored. You definitely do not want to lose this. May I suggest you reference my other blog entry here where I wrote a really simple backup/restore tool for a PostgreSQL database.
  • /etc/poweradmin/*: Your PowerAdmin flat file configuration allowing you to centrally manage everything via a webpage.
  • /etc/pdns/*: Your PowerDNS flat file configuration which defines the core of your DNS Server. It’s configuration allows you to centrally manage everything else through the PowerAdmin website.
  • /etc/pdns-recursor/*: Your PowerDNS Recursor flat file configuration which grants you the recursive functionality of your DNS Server.

What about Apache?

Apache is a perfectly fine alternative solution as well! I simply chose NginX because it is much more lightweight approach. In fact, PowerAdmin already comes with Apache configuration out of the box located in /etc/httpd/conf.d/. Thus, if you simply start up your Apache instance (service httpd start), you will be hosting its services right away. Please keep in mind that the default (Apache) configuration does not come with all the SSL and added security I provided with the NginX templates. Perhaps later on, I will update the template rpm to include an Apache secure setup as well.

Credit

This blog took me a very (,very) long time to put together and test! The repository hosting alone now accomodates all my blog entries up to this date. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

I’ve tried hard to make this a complete working solution out of the box. Please feel free to email me or post comments below with any suggestions you have so I can ensure this blog is as complete as possible! Positive feedback is always welcome too!

Repository

This blog makes use of my own repository I loosely maintain. If you’d like me to continue to monitor and apply updates as well as hosting the repository for long terms, please consider donating or offering a mirror server to help me out! This would would be greatly appreciated!

Sources