Category Archives: Linux HOWTO

Apprise Nagios Integration

Integrate Apprise into Nagios for More Notification Support

Introduction

Apprise is an open source tool that allows you to send a notification through a wide range of messaging services out there (such as Discord, Slack, Telegram, Microsoft Teams, etc). Well when you combine this with Nagios, you open it up to a much larger scope then simply emailing on an alert.

With Apprise you can configure Nagios to text your mobile phone using Amazon’s Web Service, or notify your devops team on Slack and/or Microsoft Teams. You can even trigger an IFTTT event. But it doesn’t just stop there, Apprise already supports over 35+ notification services today (and is always expanding) which means Nagios could leverage all of this too. This blog will explain how you can set up your instance of Nagios to notify more end points then just email.

The Installation

I’m going to presume you have a copy of Nagios already installed. If you don’t you can check out my blog here on how to set up your own copy with CentOS 7. Those who are not using CentOS are certainly not out of luck though, there are lots of blogs out there to get you started.

This blog will assume you have root privileges or have sudoers privileges.

Apprise can be easily added to your system through pip:

# Install Apprise onto the system currently also hosting Nagios
sudo pip install apprise

Configure Nagios

The Nagios configuration files can vary in their location depending on what Linux distribution you’re using. I’m going to just refer to some standard paths used by the stuff I host here (for CentOS/RedHat).

Step 1: Nagios Import Directory

If you’re using the Nuxref RPMs, then you can skip this step and move to the next as you’ll already be configured for this. Those using another distribution will want to update their nagios.cfg to point to a directory we can use to drop in and remove configuration from. The file is presumably going to be located as: /etc/nagios/nagios.cfg):

# Place this anywhere in /etc/nagios/nagios.cfg
# preferably put it near the bottom of the file.

# Definitions for global configuration directory
cfg_dir=/etc/nagios/conf.d

Now make sure this directory exists because this is where we’ll place our new apprise configuration:

# Ensure our global include directory exists that we
# just defined in our nagios.cfg file:
mkdir -p /etc/nagios/conf.d

# Place a dummy file in here so that Nagios doesn't
# throw any errors (as it isn't a fan of include directories
# without configuration files in it).
touch /etc/nagios/conf.d/dummy.cfg

Step 2: Apprise/Nagios Integration

Now we need to let Nagios know about Apprise. We’ll do this by creating the following files called /etc/nagios/conf.d/apprise.cfg

#
# Apprise to Nagios Configuration File
# Place this file as /etc/nagios/conf.d/apprise.cfg
#
# 'notify-host-by-apprise' command definition
define command{
   command_name   notify-host-by-apprise
   command_line   /usr/bin/printf "%b" "- *Notification Type*: $NOTIFICATIONTYPE$\n- *Host*: $HOSTNAME$\n- *State*: \n- *Address*: $HOSTADDRESS$\n- *Info*: $HOSTOUTPUT$\n\n- *Date/Time*: $LONGDATETIME$\n" | /usr/bin/apprise -c /etc/nagios/apprise.yml -n "$HOSTSTATE$" -g "$NOTIFICATIONTYPE$" -t "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **"
}
 
# 'notify-service-by-apprise' command definition
define command{
   command_name   notify-service-by-apprise
   command_line   /usr/bin/printf "%b" "*Notification Type*: $NOTIFICATIONTYPE$\n- *Service*: $SERVICEDESC$\n- *Host*: $HOSTALIAS$\n- *Address*: $HOSTADDRESS$\n- *State*: $SERVICESTATE$\n- *Date/Time*: $LONGDATETIME$\n\n*Additional Info*:\n$SERVICEOUTPUT$\n" | /usr/bin/apprise -c /etc/nagios/apprise.yml -n "$HOSTSTATE$" -g "$NOTIFICATIONTYPE$" -t "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **"
}

# Register our contact template that we can reference
define contact{
  ; The name of this contact template
  name                             apprise-contact

  ; service notifications can be sent anytime
  service_notification_period      24x7

  ; host notifications can be sent anytime
  host_notification_period         24x7

  ; send notifications for all service states, flapping events,
  ; and scheduled downtime events
  service_notification_options     w,u,c,r,f,s

  ; send notifications for all host states, flapping events,
  ; and scheduled downtime events
  host_notification_options        d,u,r,f,s

  ; send service notifications via email
  service_notification_commands    notify-service-by-apprise

  ; send host notifications via email
  host_notification_commands       notify-host-by-apprise

  ; Don't register this as it is just a template for future
  ; references by contacts who wish to use the apprise plugin
  register                         0
}

Now for every contact we set up going forward, we can point it to use Apprise. By default Nagios usually provides us a contact.cfg file that contains the generic user nagiosadmin. For those using my packaging, you can find this file at /etc/nagios/objects/contacts.cfg; you’ll want to change it to looks like this:

define contact{
  ; Short name of (Nagios) user
  contact_name    nagiosadmin

  ; This next line used to read generic-contact; but we want to switch it
  ; over to our new Apprise based one:
  use             apprise-contact

  ; Full name of user
  alias           Nagios Admin

  ; not important if using apprise-contact (defined above)
  email           nagios@localhost
}

Before you advance to the next step, you’ll want to run a test flight check on your configuration and make sure it validates okay.

# Perform a flight check on our new configuration (as root)
sudo nagios -v /etc/nagios/nagios.cfg

If you get any errors, you should revisit the first part of this blog and try to iron them out before continuing. If everything is error free, then the next step is to reload our instance of Nagios (if it’s running) so it can re-read this configuration. This can be done with the command:

# You will need to be root to do this; send a SIGHUP
# to all instances of nagios running in memory:
sudo killall -HUP nagios

Step 3: Apprise Configuration

Now we need to prepare our Apprise configuration (/etc/nagios/apprise.yml) and fill it with the notification services we want listen for and who we want to pass it to.

We can associate with Nagios notifications types passed to us through tags. Nagios will pass these along one of the following $NOTIFICATIONTYPE$ when an event occurs; these are:

  • PROBLEM: There was an issue with one of the checks.
  • RECOVERY: The issue previously set has been cleared.
  • ACKNOWLEDGEMENT: An outstanding issue has been acknowledged.
  • FLAPPINGSTART: Flapping is a state where a service has a PROBLEM associated with it and then moments later has a RECOVERY. This is the state called FLAPPING. When this process occurs too many times in a row, this alert gets set.
  • FLAPPINGSTOP: The service that was previously FLAPPING is no longer doing so.
  • FLAPPINGDISABLED: Someone just disabled FLAPPING for this service/host.
  • DOWNTIMESTART: The scheduled downtime for this service/host has begun.
  • DOWNTIMEEND: The scheduled downtime is over.
  • DOWNTIMECANCELLED: Someone just cancelled the scheduled downtime for this service/host.

Knowing the above notification types that we’ll receive, here is what an Apprise configuration file located at /etc/nagios/apprise.yml might look like:

# This file should be placed in /etc/nagios/apprise.yml

# NOTE: THIS IS JUST AN EXAMPLE CONFIGURATION FILE. YOU WILL WANT
#       TO CUSTOMIZE YOUR OWN WITH THE SERVICE(S) OF YOUR CHOICE
#       VISIT https://github.com/caronc/apprise TO SEE WHAT IS
#       AVAILABLE AND HOW THEY WORK.

# Identify all of the global notification types we want to flag on.
tag:
  - PROBLEM
  - RECOVERY
  - FLAPPINGSTART
  - FLAPPINGSTOP

# Now we want to define our Apprise URLS; you'll want to visit
# https://github.com/caronc/apprise to see all of the supported
# services and how to build their URLs.
urls:

  # Maybe we want to notify a custom service we're hosting to
  # monitor and track Nagios status; Check out the following 
  # for more details https://github.com/caronc/apprise/wiki/Notify_json

  - json://localhost

  # Maybe we want to notify a Slack channel; more details on this
  # are here: https://github.com/caronc/apprise/wiki/Notify_slack
  - slack://T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F/#nuxref

  # the Apprise YAML configuration is quite powerful, the
  # following prepares the email URL and sends an email to each
  # user identified below:
  - email://user:password@gmail.com
      - to: george@example.com
      - to: admin@example.com

  # More details on the emails can be found here:
  # https://github.com/caronc/apprise/wiki/Notify_email

  # We can also individually disperse the tags in the same config
  # file.  The below tags will override the globals defined above.
  # A use case for this would be that maybe we just want to
  # send certain notification types to say... the DevOps team:
  - email://user:password@gmail.com
      - to: devops@example.com
        tag: DOWNTIMESTART, DOWNTIMEEND, DOWNTIMECANCELLED

Once you’re file is all ready, be sure this file is readable by Nagios (to keep it away from prying eyes), but otherwise you’re all set and ready to go!

# Here is what one might do to protect this apprise configuration
chmod 640 /etc/nagios/apprise.yml
chown nagios.root /etc/nagios/apprise.yml

Verification

To be sure everything works, you may want to just test that you got all of your configuration right You can test this using manually as follows:

# Test our configuration with apprise using the PROBLEM tag
# -vvv for some verbose debugging in-case we need it.
apprise -c /etc/nagios/apprise.yml \
    -n CRITICAL -g PROBLEM \
    -vvv \
    -t "A Test Title" \
    -b "a Test Body"

Here is a screenshot of a test error displayed on gitter.im that was sent by Nagios using Apprise:
Gitter Example

Sources

An Inexpensive and Reliable VOIP Solution

Introduction

In today’s times, most people cancel their land lines because it’s getting just too expensive to have one. I mean cell phone costs are expensive enough and that’s usually the best way to get a hold of someone these days anyway. But there are times where a land line is nice to have too. I mean it’s much nicer giving your relatives one phone number to reach your whole family instead of giving them a list of cell phone #’s.

But what if there was a solution that would make it worth while? Right now, I’m paying slightly less than $3/month (USD) for my land line and here is what I get:

  • World Wide (Cost Effective) Calling: There is really no more such thing as long distance anymore. It costs less than a penny to call anywhere in the world (literally!).
  • Call Display: That handy feature that lets us see who’s calling before we answer the phone.
  • Call Answer: An automated message when you’re not home giving the caller the chance to leave a message.
  • Call Waiting: That handy feature that lets a second call come in while you’re already on the phone.
  • 911 Service: This actually costs $1.50 (USD) of your monthly fee; I’m already including it in my fee mentioned above. If you feel you don’t need it, then your bill will be even cheaper then mine.

I realize that you probably get all this with your own personal phone plan(s) too; but keep in mind that we’re still only talking about $3/month (USD) here. Here is what else you can do with the VOIP provider discussed in this blog (at no extra costs):

VOIP.ms Android Management App

VOIP.ms Android Management App

  • Voicemail Emailed To You: You can have your phone messages (from the Call Answer feature) emailed to you as a small sound clip once one is left. You can get your home messages from anywhere as soon as their left! This eliminates that hassle from having to log in and check your voice mail every time someone leaves one.
  • True Do Not Call (DNCL) Lists: I’m not sure about you, but The National Do Not Call List (for Americans or Canadians) doesn’t work the greatest. We still continue to get Telemarketers calling from oversees and others who violate this agreement simply because they’re calling from a country that just refuses to follow our regulations. Well with the VOIP setup I explain in this blog, you can create a real “Do Not Call” filter by entering these people into a list (that you maintain) blocking them from ever calling again.
  • SMS Messaging: You can send Text Messages to and from your land line. This is really handy! You can use this Android App to do it with too!
  • Advanced Call Forwarding:You can have you phone calls forwarded to another phone at any time. You can also just choose to set up a forward on certain people who call.
  • Transparency & Control: Using the VOIP.ms Android Management App you can:
    • Access your current bills at any time.
    • View your missed calls.
    • Change your settings (forwarding, call answer, etc)
    • Block spammers
    • And much… much more!

Hopefully I’ve still got your attention because this truly is a very frugal but amazing phone (land line) solution!

Requirements

So here is what you need:

  • Any Computer: The only role this computer plays is during the initial setup (account setup, device configuration, etc).
  • A VOIP Telephone: Basically any phone that does not draw it’s primary source of power from the phone line. DECT phones are the cheapest solution and what I strongly recommend you get (which is basically any cordless phone). Don’t worry if you’re confused here because just about any phone (set) that has to plug into an electrical outlet in addition of the phone connection will work for you here.
    VOIP Compatible Phone

    VOIP Compatible Phone

  • An Analog Telephone Adapter (ATA): This is the key component of your setup! It is the core device that will translate all of your incoming and outgoing calls via the internet to your home phone. Again, don’t get confused here; I personally use the GrandStream GS-HT702 and will base this entire blog on it.
    This will be be one of your only one-time up-front cost that will set you back about $40 USD. I imagine most of you reading this blog will probably have to go out and buy this and then come back later finish your hookup. You can find this on Amazon, Home Depot, etc…
  • A VoIP.ms Account: Here is a link to their site. Your ATA device will communicate with your VoIP.ms account and allow you to make calls and receive calls. It’s VoIP.ms that you’ll be paying the $1 to $3 (USD) a month to for your service.
  • An Internet Account: It’s the 21st century so I’m just going to go ahead and assume you’ve already got one.

voip-setup
Depending on your budget, it wouldn’t hurt to buy a UPS device as well. This is totally optional, but the one thing with a VOIP solution is that it requires the internet at all times to function. If you get bad weather in your area and lose power to your home, then (of course) you lose your network/internet connection as well. A UPS device however solves this potential dilemma allowing you to always have (both your internet and) your VOIP phone service.

VOIP In a Nutshell

Before we get any further in this blog, we should identify a few acronyms that will show up throughout the setup screens on your ATA’s configuration. You can skip over this section if you wish, but feel free to refer to it late if something confuses you. It will also make your VOIP experience easier too.

VOIP Acronyms
VOIP Voice Over IP: Just thought I’d get the most obvious acronym out of the way. This is the ability to make phone calls over the internet using a common phone in your house.
DID Direct Inward Dialing: This is referring to your 7 digit phone number ((XXX) XXX-XXXX) that you will use to make and receive calls on. If you already have a land line # you want to use; this isn’t a problem at all. All VOIP (SIP) Providers (including VoIP.ms) will allow you to transfer your existing number over to them.
POP Point of Presence: Not to be confused with emails post office protocol; this acronym (with respect to VOIP) takes on a different meaning entirely. The Point of Presence identifies the server you will receive and handle all of your incoming phone calls at; voice mail gets stored here too. When using VoIP.ms as your provider; this is also the server that your outgoing calls will pass through.
SIP Session Initiation Protocol: You will see this acronym come up a lot. This is the language (or protocol) your GrandStream (ATA) device speaks to your VIP Provider with. It’s the language makes it possible to transmit voice over the internet. The POP Server will communicate via SIP. There are times when you’ll need to provide your SIP Server; what they’re really just asking for here is your POP Server (identified above).
ATA Analog Telephone Adapter: This refers to the device that communicates with the POP server using SIP. It relays the voices to and from your home phone to your VoIP provider. This is a major piece of the puzzle that connects everything together. For this blog the GrandStream HandyTone analog telephone adapter (ATA) plays this role for us.
FXS Foreign eXchange Subscriber: Your ATA device (in our case the GrandStream HandyTone) will associate each (phone) jack it has (that you can plug your phones into). Although our ATA device has more the one connector, we only need to configure one of them.

Now that you know the acronyms applicable to what we’re trying to accomplish; lets illustrate it so you see how easy it really is:
Acronym Buster

Setting Up Your VOIP Account

  1. The first thing you’re going to want to do is create an account with https://www.voip.ms.
  2. Once you’ve got an account; You must set up your Direct Inward Dialing (DID) Number. This involves the transfer (or creation) of a 7 digit phone number people can reach you at. It’s also the number you will make your outgoing calls from. If you already have a phone number from an existing land line, it can be easily transferred so you don’t have to give everyone you know a new number.
  3. Next you’ll have to choose a phone plan; there are 2 main plans offered:

    1. Fixed Rate: Pay just under $5 USD a month for unlimited international calling anywhere.
    2. Variable Rate: Pay $0.85 cents (USD) a month and then pay ~0.005 cents per minute. I didn’t get those decimals wrong either; it’s just half a cent a minute!

    I personally use the Variable Rate only because you’d have to talk more then 880 minutes (($4.95 – 0.85)/0.005) a month just to match that of the unlimited plan (which I don’t do). But your mileage may vary depending on your own personal phone usage. Regardless of what choice you make, you’ll still be saving money when comparing these kind of costs to the competitors! You can also change your plan at any time too.

  4. Once you get your DID in place, you must associate it with one of the many POP servers VoIP.ms has. This process is done from their website (you must be logged in). You can configure all of this from the [DID Numbers] -> [Manage DID(s)] section of their website. You should select a server that is close to your location (to minimize on latency).

    “Latency is very important for Voip, this will determine the time that will take for the data package transmission to reach the destination. A high latency will lead to a delay and echoes in the communication.”
    – Voip.ms

    Here is a list of servers to choose from. The POP server is where all your phone calls will route from. It is also where phone messages will be left (if anyone leaves one).

Setting Up Your GrandStream (ATA) Device

The first time you open up your GrandStream Device, you’ll need to configure it. The easiest way to do this is to plug it in and hook a network cable up from it to your Linux Server. If you look underneath the GrandStream device (where the serial #’s are), you’ll see the devices MAC address (format XX:XX:XX:XX:XX:XX). The following will allow you assign it an IP address so that you can access it. My local network was 192.168.2.x. So in my case, I used .3 because I knew it was free on my network; you may wish to :

# Assign an IP Address to your ATA Device (with Linux/Mac):
sudo arp -s 192.168.2.3 00:0B:82:81:E3:BB

It’s a bit more tricky for Microsoft users, but the following should add your mapping (after you open up a shell window):

netsh -c interface ipv4 add neighbors "Local Area Connection" "192.168.2.3" "00-0B-82:81-E3-BB"

At this point you should be able to access your ATA (GrandStream) device at the IP address you specified (from the machine you mapped it’s MAC address from). In my case, I’d point my web browser to: https://192.168.2.3/

Default Credentials
Admin Password admin
User Password 123

Login to the device with the Admin account (password is admin). The setup of this device can be a bit over-whelming, but bear with me; it’s worth it.

To start off with, you might want to click on the [Basic Settings] and configure your IP Address information. This will allow your device to be accessible from other PC’s in your house.

Now for the [FXS Port1] tab; the full settings can be found here. I’ll provide mine too should that be more helpful.
A point up front:

  • Your Primary SIP Server and Outbound Proxy should be set to the POP server your configured with your VOIP.ms account.

Below identifies the rest of the configuration; I realize that the configuration presented below can seem a bit overwhelming. But remember: you only have to do this part once and there are actually very little fields that require changing!

VOIP Solution - FXS Port1 - Page 1 of 5
VOIP Solution - FXS Port1 - Page 2 of 5
VOIP Solution - FXS Port1 - Page 3 of 5
VOIP Solution - FXS Port1 - Page 4 of 5
VOIP Solution - FXS Port1 - Page 5 of 5

Troubleshooting

Grandstream Expected LightsWhen you’ve set everything up, you should see the following lights on your Grandstream Device. Unless you see all 4 lights; you will ‘not’ be able to send/receive calls. Here are some hints if you don’t see the 4 lights identified:

  • No lights?: You might have a defective device. Double check that you’ve plugged in it.
  • No Internet light?: Check your IP settings on the [Basic Settings] from within the Grandstream configuration (webpage). If you’re plugged into a wireless router or your ISP’s modem, make sure you’ve selected dynamically assigned via DHCP.
  • No Link/Act Light?: You’re not able to communicate with Outbound Proxy or Primary SIP Server defined in the [FXS PORT1] from within the Grandstream configuration (webpage). Double check that you’ve got these right!
  • No Phone1 light?: The SIP User ID you specified could not be verified/validated by your VOIP provider. You’ll need to double check it. You may also need to make sure it’s activated too!.

Sources

  • VoIP.ms: The best VOIP solution I could find! Far cheaper and more feature rich than the alternatives (such as magicJack, Ooma, Vonage, etc…)!
  • VoIP.ms POP Server Listing: If you do decide to go with these guys, you’ll want to point your ATA device to a server close to you. This is the server listing you can choose from.
  • The National Do Not Call List (DNCL) (for Americans or Canadians: Even if you don’t plan on getting VOIP as described in this blog. You might want to at least add your cell phones to this list. It isn’t perfect, but it’s better than nothing!
  • GrandStream GS-HT702: This is the ATA device I personally use. This is also the device my blog focuses specifically.
  • GrandStream GS-HT702 VOIP Setup: Specifically the setup for VoIP.ms.
  • VoIP.ms Console: A fantastic Android App that lets you manage your VoIP.ms account from your mobile device (or tablet).
  • VoIP.ms SMS: Yet another fantastic Android App that lets you send and receive SMS (text) messages through your land line via your mobile device (or tablet).

Just for the record: VoIP.ms does not have a referral program (for me to benefit from), nor do I work for them at all. I speak highly of them because their pricing is incomparable to any of their competitors (in the best way possible), and their support is absolutely amazing. I’ve been using VoIP.ms for just over a year now and have been so happy with my service that I felt like blogging about it.

Nov 19th, 2018 Update: VoIP.ms now has a referral program! 🙂

AWStats Setup on CentOS

Introduction

AWStats is a great tool for gathering statistics about your website. It acquires everything it needs to know about your site strictly through your websites log files. AWStats is able to scan through these logs line by line and present them in a fantastic report. This report can really help you make strategic decisions going forward as well as spot any anomalies that might be taking place. The tool is smart enough to only scan newer log entries (from when it last ran) allowing you to run it again and again (as often as you want). Thus, once you set this tool up to run daily (or even hourly), you’ll have detailed statistics about your website you can call upon anytime.

AWStats collects information such as such as:AWStats Report

  • Who is visiting your site.
  • How many visitors you’re getting daily.
  • Where are they’re coming from (did a site link to you?)
  • Where is the visitor from (geographical location
  • … and on and on
  • The presentation of these collected statistics can be either via a website (HTML), XML and/or as a PDF file. The PDF is especially useful since it combines all of the multiple HTML pages (as presented) into one great big report with a table of contents and hyperlinks throughout it! The PDF is also really easy to navigate and pass along to others who might also be interested.

    Why Use AWStats over Google Analytics?

    Google Analytics Inaccuracy

    Google Analytics Inaccuracy

    The number one reason is because AWStats is much (,much) more accurate! AWStats also just works without ‘any’ changes to your website (literally – none at all). Google Analytics however requires you to add a small piece of JavaScript to every web page you want to track. Every time this tiny bit of JavaScript code executes, it passes the information along to Google. The problem is… if that little snippet of JavaScript doesn’t execute, then Google doesn’t track that user (and you’ll never know) because it just won’t get reported.

    It’s really easy to prevent this chunk of JavaScript from running too, you just have to have installed something like Ad-Blocker Plus, Disconnect and/or uBlock into your Web Browser (such as Firefox or Chrome). These plugins specifically block these tracking techniques and eliminate most (if not all) advertising the website might have too.

    It doesn’t mean that online analytic tools (like Google Analytics) are not good; no, not at all! But it’s just important to understand that they can’t (and truly aren’t) reporting everything that’s going on with your website and the traffic generated from it.

    Another point worth mentioning is that Google Analytics can not monitor and report statistics on traffic used by third party tools. Therefore you can’t use it to monitor any RESTful API services because the programs accessing it will never call these JavaScript snippets of code.

    It’s worth pointing out now that if you use AWStats, you’ll have the full picture! You’ll be able to easily identify any anomalies and detect certain forms of malicious intent! You’ll be able to monitor all of your internal (web based) services you may manage. From the public standpoint, you might be very surprised at how much more traffic your website is getting despite what online analytic tools will tell you!

    Let’s Get Started

    First you’ll want to install the proper packages. You should hook up to my repository and the EPEL repository as well! The EPEL repository hosts AWStats too, but mine is a newer version. We need the EPEL repository for it’s GeoIP packages since they get updated more often there:

    # CentOS 7 users can connect to EPEL this way:
    rpm -Uhi https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    
    # Similarly, you can hook up to my repository at http://nuxref.com
    # but here is a quick way of doing it (for CentOS/RedHat 7):
    rpm -Uhi http://repo.nuxref.com/centos/7/en/x86_64/custom/nuxref-release-1.0.0-4.el7.nuxref.noarch.rpm
    

    You should be good to go now; the following installs AWStats and a few extra tools to get the best out of it:

    # install awstats
    # install htmldoc too because it'll allow you to create a pdf
    # install geoip-geolite for the ability to track the IPs
    #        to countries
    # install perl-Geo-IP to look up the IP Addresses
    yum install awstats htmldoc geoip-geolite perl-Geo-IP
    
    

    AWStat In a Nutshell

    The steps below will require that you have set up the environment defined below. Obviously you’ll want to change these environment variables to suite your own needs:

    # First define our website as a variable.
    # We will use this value to track and store in an
    # organized structure.
    
    # Those who host other websites for people can
    # change this and virtually everything below will
    # and re-run everything to get stats for that too!
    WEBSITE=nuxref.com
    
    # AWSTATS Variable Data
    DATADIR=/var/lib/awstats/$WEBSITE
    
    

    Configuring AWStats: Step 1 of 3

    AWStats works from configuration files you create in /etc/awstats/. But it also needs a directory it can work within (we use /var/lib/awstats/). I’ve provided documentation around each line so you know what’s going on:

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    
    # First we need to setup our DATADIR; this is where
    # all our statistics and generated data will be placed
    # into:
    [ ! -d $DATADIR/static ] && \
        mkdir -p $DATADIR/static
    ln -snf /usr/share/awstats/wwwroot/icon \
       $DATADIR/static/icon
    ln -snf /usr/share/awstats/wwwroot/cgi-bin \
       $DATADIR/static/cgi-bin
    
    # Create a configuration file using our website
    # based on the awws.model.conf example file that
    # ships with AWStats
    sed -e "s|localhost\.localdomain|$WEBSITE|g" \
    	/etc/awstats/awstats.model.conf > \
    		/etc/awstats/awstats.$WEBSITE.conf
    
    ########################################
    # Now update our new configuration
    ########################################
    # Update the LogFile with our access.log file we'll
    # reference. This path doesn't exist yet but
    # we'll be creating it soon enough; leave this entry
    # untouched (don't change it to your real log path!):
    sed -i -e "s|^\(LogFile\)=.*$|\1=\"$DATADIR/access.log\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    # Disable DNS (for speed mostly)
    sed -i -e "s|^\(DNSLookup\)=.*$|\1=0|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    # For PDF Generation we need to update the relative
    # paths for the icons.
    sed -i -e "s|^\(DirIcons\)=.*$|\1=\"icon\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    sed -i -e "s|^\(DirCgi\)=.*$|\1=\"cgi-bin\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    

    Optionally Configuring GeoIP Updates

    The geolite data fetches us a great set of (meta) data we can reference when looking up IP Addresses (of people who visited our site) and determining what part of the world they came from. This information is fantastic when putting together statistics and web page traffic like AWStats does.

    First we want to configure AWStats to use the GEO IP Plugin:

    # Now configure our GEOIP Setup
    sed -i -e '/^LoadPlugin=.*/d' /etc/awstats/awstats.$WEBSITE.conf
    cat << _EOF >> /etc/awstats/awstats.$WEBSITE.conf
    LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"
    LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /usr/share/GeoIP/GeoIPCity.dat"
    _EOF
    

    Next we want to set up our GEO IP to update itself with the latest meta data for us automatically (so we don’t have to worry about it):

    # downloads all of the latest GEO IP content to
    # /usr/share/GeoIP with this simple command:
    geoipupdate
    
    # This IP information changes often; so the next
    # thing you want to do is create a cronjob to have
    # this tool fetch regular updates automatically for
    # us to keep the GEO IP Content fresh and up to date!
    cat << _EOF > /etc/cron.d/geoipdate
    0 12 * * 3 root /usr/bin/geoipupdate &>/dev/null
    _EOF
    

    Apache Users: Step 2a of 3

    AWStats depends on the log files to build it’s statistics from, so it’s important we point it to the right directory. Apache logs have been pretty much standardized and AWStats just works with them. If your web page is being hosted through Apache then your log files are most likely being placed in /var/log/httpd. If you’re using NginX (and not Apache), you can skip over this section and to Step 2b of 3 instead.

    Make sure AWStats knows it’s dealing with Apache log files (make sure you’ve still got the $WEBSITE variable defined from above):

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    ########################################
    # Apache Users Should run The Following
    ########################################
    # Now if you're logs are created from Apache you
    # need to run the following:
    # Log Format (Type 1 is for Apache)
    sed -i -e "s|^\(LogFormat\)=.*$|\1=1|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    

    Now what we want to do is take all of the logs files associated with our website in /var/log/httpd and build one great big (sorted) log file we can get all of our statistics out of:

    # logresolvmerge.pl is a fantastic tool that ships with
    # awstats and merges (and sorts) all of our logs. We
    # place the output into our $DATADIR (which we declared
    # earlier):
    /usr/share/awstats/tools/logresolvemerge.pl \
       /var/log/httpd/access.log \
       /var/log/httpd/access.log-????????.gz \
        > $DATADIR/access.log
    
    

    Nginx Users

    NginX logs have a slightly different format then the Apache logs and therefore require a slightly different configuration to work. If your web page is being hosted through NginX then your log files are most likely being placed in /var/log/nginx. If you’re using Apache (and not NginX), then you can skip over this section as long as you’ve already done Step 2a of 3 instead.

    Make sure AWStats knows it’s dealing with NginX log files otherwise it won’t be able to interpret them. Also be sure to have your $WEBSITE variable defined:

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    ########################################
    # NginX Users Should run The Following
    ########################################
    # If you're using NginX, you'll want to adjust
    # your awstat LogFormat entry as follows:
    sed -i -e "s|^\(LogFormat\)=.*$|\1=\"%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    

    Now we take all of the logs files associated with our website in /var/log/nginx and build one great big (sorted) log file we can get all of our statistics out of:

    # logresolvmerge.pl is a fantastic tool that ships with
    # awstats and merges (and sorts) all of our logs. We
    # place the output into our $DATADIR (which we declared
    # earlier):
    /usr/share/awstats/tools/logresolvemerge.pl \
       /var/log/nginx/access.log \
       /var/log/nginx/access.log-????????.gz \
        > $DATADIR/access.log
    

    Statistic Generation: Step 3 of 3

    At this point we have all the info we need

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    ########################################
    # (Create) and/or Update our Stats
    ########################################
    /usr/share/awstats/wwwroot/cgi-bin/awstats.pl \
       -config=$WEBSITE
    
    # The following builds us a PDF file containing all
    # of our statistics in addition to a website we can
    # optionally host if we want.
    # The following would allow you to gather statistics for
    # a given year:
    #   /usr/share/awstats/tools/awstats_buildstaticpages.pl \
    #      -config=$WEBSITE -buildpdf \
    #      -month=all -year=$(date +'%Y') \
    #      -dir=$DATADIR/static \
    #      -buildpdf=/usr/bin/htmldoc
    
    # This will build statistics with all the information we have:
    /usr/share/awstats/tools/awstats_buildstaticpages.pl \
       -config=$WEBSITE -buildpdf \
       -dir=$DATADIR/static \
       -buildpdf=/usr/bin/htmldoc
    
    # - The main website will appear as:
    #      $DATADIR/static/awstats.$WEBSITE.html
    #    But this 'main' website links to several other websites
    #    that can also all be found in the $DATADIR/static
    #    directory
    # - The pdf file will appear as:
    #      $DATADIR/static/awstats.$WEBSITE.pdf
    

    Consider throwing the above into a script file and having it ran in a cron job!

    Hosting The Statistics

    This option is purely optional; but but here is some simple configurations you can use if you want to access these generated statistics from your browser.

    Note: I intentionally keep things simple in this section. AWStats can be configured so that you can update your statistics via it’s very own website (see AllowToUpdateStatsFromBrowser directive in the site configuration). However I don’t recommend this option and therefore do not document it below.

    NginX

    A simple NginX configuration might look like this:

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    cat << _EOF > /etc/nginx/default.d/awstats.$WEBSITE.conf
       # Visit your statistics by browsing to:
       # if WEBSITE was equal nuxref.com, you'd visit the stats:
       # http://localhost/stats/nuxref.com/
       location /stats/$WEBSITE/ {
          alias   $DATADIR/$WEBSITE/static/;
          index  awstats.$WEBSITE.html;
    
          ## Set 1.2.3.4 to your own IP address and uncomment
          ## the entries below to 'only' allow yourself access to
          ## these stats:
          # allow 1.2.3.4/32;
          # deny all;
    
          location /stats/css/ {
              alias /usr/share/awstats/wwwroot/css/;
          }
    
          location /stats/icon/ {
              alias /usr/share/awstats/wwwroot/icon/;
          }
       }
    _EOF
    

    Don’t forget to reload NginX so it takes on your new configuration (and makes that statistics page visible):

    # Reload NginX
    systemctl reload nginx.service
    

    Apache

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    cat << _EOF > /etc/httpd/conf.d/awstats.$WEBSITE.conf
       # Visit your statistics by browsing to:
       # if WEBSITE was equal nuxref.com, you'd visit the stats:
       # http://localhost/stats/nuxref.com/
       Alias /stats/$WEBSITE/ "$DATADIR/$WEBSITE/static/"
       <Directory "$DATADIR/$WEBSITE/static/">
          Options FollowSymLinks
          AllowOverride None
          Order allow,deny
          Allow from all
    
          ## Set 1.2.3.4 to your own IP address and uncomment
          ## the entries below to 'only' allow yourself access to
          ## these stats:
          # Order deny,allow
          # Deny from all
          # Allow from 1.2.3.4/255.255.255.255
       </Directory>
    _EOF
    

    Don’t forget to reload Apache so it takes on your new configuration (and makes that statistics page visible):

    # Reload Apache
    systemctl reload httpd.service
    

    Credit

    This blog took me a long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. I took the open source available to me and rebuilt it to make it an easier solution and decided to share it. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

    Sources