Category Archives: Linux HOWTO

An Inexpensive and Reliable VOIP Solution

An Inexpensive and Reliable VOIP Solution

Introduction

In today’s times, most people cancel their land lines because it’s getting just too expensive to have one. I mean cell phone costs are expensive enough and that’s usually the best way to get a hold of someone these days anyway. But there are times where a land line is nice to have too. I mean it’s much nicer giving your relatives one phone number to reach your whole family instead of giving them a list of cell phone #’s.

But what if there was a solution that would make it worth while? Right now, I’m paying slightly less than $3/month (USD) for my land line and here is what I get:

  • World Wide (Cost Effective) Calling: There is really no more such thing as long distance anymore. It costs less than a penny to call anywhere in the world (literally!).
  • Call Display: That handy feature that lets us see who’s calling before we answer the phone.
  • Call Answer: An automated message when you’re not home giving the caller the chance to leave a message.
  • Call Waiting: That handy feature that lets a second call come in while you’re already on the phone.
  • 911 Service: This actually costs $1.50 (USD) of your monthly fee; I’m already including it in my fee mentioned above. If you feel you don’t need it, then your bill will be even cheaper then mine.

I realize that you probably get all this with your own personal phone plan(s) too; but keep in mind that we’re still only talking about $3/month (USD) here. Here is what else you can do with the VOIP provider discussed in this blog (at no extra costs):

VOIP.ms Android Management App

VOIP.ms Android Management App

  • Voicemail Emailed To You: You can have your phone messages (from the Call Answer feature) emailed to you as a small sound clip once one is left. You can get your home messages from anywhere as soon as their left! This eliminates that hassle from having to log in and check your voice mail every time someone leaves one.
  • True Do Not Call (DNCL) Lists: I’m not sure about you, but The National Do Not Call List (for Americans or Canadians) doesn’t work the greatest. We still continue to get Telemarketers calling from oversees and others who violate this agreement simply because they’re calling from a country that just refuses to follow our regulations. Well with the VOIP setup I explain in this blog, you can create a real “Do Not Call” filter by entering these people into a list (that you maintain) blocking them from ever calling again.
  • SMS Messaging: You can send Text Messages to and from your land line. This is really handy! You can use this Android App to do it with too!
  • Advanced Call Forwarding:You can have you phone calls forwarded to another phone at any time. You can also just choose to set up a forward on certain people who call.
  • Transparency & Control: Using the VOIP.ms Android Management App you can:
    • Access your current bills at any time.
    • View your missed calls.
    • Change your settings (forwarding, call answer, etc)
    • Block spammers
    • And much… much more!

Hopefully I’ve still got your attention because this truly is a very frugal but amazing phone (land line) solution!

Requirements

So here is what you need:

  • Any Computer: The only role this computer plays is during the initial setup (account setup, device configuration, etc).
  • A VOIP Telephone: Basically any phone that does not draw it’s primary source of power from the phone line. DECT phones are the cheapest solution and what I strongly recommend you get (which is basically any cordless phone). Don’t worry if you’re confused here because just about any phone (set) that has to plug into an electrical outlet in addition of the phone connection will work for you here.
    VOIP Compatible Phone

    VOIP Compatible Phone

  • An Analog Telephone Adapter (ATA): This is the key component of your setup! It is the core device that will translate all of your incoming and outgoing calls via the internet to your home phone. Again, don’t get confused here; I personally use the GrandStream GS-HT702 and will base this entire blog on it.
    This will be be one of your only one-time up-front cost that will set you back about $40 USD. I imagine most of you reading this blog will probably have to go out and buy this and then come back later finish your hookup. You can find this on Amazon, Home Depot, etc…
  • A VoIP.ms Account: Here is a link to their site. Your ATA device will communicate with your VoIP.ms account and allow you to make calls and receive calls. It’s VoIP.ms that you’ll be paying the $1 to $3 (USD) a month to for your service.
  • An Internet Account: It’s the 21st century so I’m just going to go ahead and assume you’ve already got one.

voip-setup
Depending on your budget, it wouldn’t hurt to buy a UPS device as well. This is totally optional, but the one thing with a VOIP solution is that it requires the internet at all times to function. If you get bad weather in your area and lose power to your home, then (of course) you lose your network/internet connection as well. A UPS device however solves this potential dilemma allowing you to always have (both your internet and) your VOIP phone service.

VOIP In a Nutshell

Before we get any further in this blog, we should identify a few acronyms that will show up throughout the setup screens on your ATA’s configuration. You can skip over this section if you wish, but feel free to refer to it late if something confuses you. It will also make your VOIP experience easier too.

VOIP Acronyms
VOIP Voice Over IP: Just thought I’d get the most obvious acronym out of the way. This is the ability to make phone calls over the internet using a common phone in your house.
DID Direct Inward Dialing: This is referring to your 7 digit phone number ((XXX) XXX-XXXX) that you will use to make and receive calls on. If you already have a land line # you want to use; this isn’t a problem at all. All VOIP (SIP) Providers (including VoIP.ms) will allow you to transfer your existing number over to them.
POP Point of Presence: Not to be confused with emails post office protocol; this acronym (with respect to VOIP) takes on a different meaning entirely. The Point of Presence identifies the server you will receive and handle all of your incoming phone calls at; voice mail gets stored here too. When using VoIP.ms as your provider; this is also the server that your outgoing calls will pass through.
SIP Session Initiation Protocol: You will see this acronym come up a lot. This is the language (or protocol) your GrandStream (ATA) device speaks to your VIP Provider with. It’s the language makes it possible to transmit voice over the internet. The POP Server will communicate via SIP. There are times when you’ll need to provide your SIP Server; what they’re really just asking for here is your POP Server (identified above).
ATA Analog Telephone Adapter: This refers to the device that communicates with the POP server using SIP. It relays the voices to and from your home phone to your VoIP provider. This is a major piece of the puzzle that connects everything together. For this blog the GrandStream HandyTone analog telephone adapter (ATA) plays this role for us.
FXS Foreign eXchange Subscriber: Your ATA device (in our case the GrandStream HandyTone) will associate each (phone) jack it has (that you can plug your phones into). Although our ATA device has more the one connector, we only need to configure one of them.

Now that you know the acronyms applicable to what we’re trying to accomplish; lets illustrate it so you see how easy it really is:
Acronym Buster

Setting Up Your VOIP Account

  1. The first thing you’re going to want to do is create an account with https://www.voip.ms.
  2. Once you’ve got an account; You must set up your Direct Inward Dialing (DID) Number. This involves the transfer (or creation) of a 7 digit phone number people can reach you at. It’s also the number you will make your outgoing calls from. If you already have a phone number from an existing land line, it can be easily transferred so you don’t have to give everyone you know a new number.
  3. Next you’ll have to choose a phone plan; there are 2 main plans offered:

    1. Fixed Rate: Pay just under $5 USD a month for unlimited international calling anywhere.
    2. Variable Rate: Pay $0.85 cents (USD) a month and then pay ~0.005 cents per minute. I didn’t get those decimals wrong either; it’s just half a cent a minute!

    I personally use the Variable Rate only because you’d have to talk more then 880 minutes (($4.95 – 0.85)/0.005) a month just to match that of the unlimited plan (which I don’t do). But your mileage may vary depending on your own personal phone usage. Regardless of what choice you make, you’ll still be saving money when comparing these kind of costs to the competitors! You can also change your plan at any time too.

  4. Once you get your DID in place, you must associate it with one of the many POP servers VoIP.ms has. This process is done from their website (you must be logged in). You can configure all of this from the [DID Numbers] -> [Manage DID(s)] section of their website. You should select a server that is close to your location (to minimize on latency).

    “Latency is very important for Voip, this will determine the time that will take for the data package transmission to reach the destination. A high latency will lead to a delay and echoes in the communication.”
    – Voip.ms

    Here is a list of servers to choose from. The POP server is where all your phone calls will route from. It is also where phone messages will be left (if anyone leaves one).

Setting Up Your GrandStream (ATA) Device

The first time you open up your GrandStream Device, you’ll need to configure it. The easiest way to do this is to plug it in and hook a network cable up from it to your Linux Server. If you look underneath the GrandStream device (where the serial #’s are), you’ll see the devices MAC address (format XX:XX:XX:XX:XX:XX). The following will allow you assign it an IP address so that you can access it. My local network was 192.168.2.x. So in my case, I used .3 because I knew it was free on my network; you may wish to :

# Assign an IP Address to your ATA Device (with Linux/Mac):
sudo arp -s 192.168.2.3 00:0B:82:81:E3:BB

It’s a bit more tricky for Microsoft users, but the following should add your mapping (after you open up a shell window):

netsh -c interface ipv4 add neighbors "Local Area Connection" "192.168.2.3" "00-0B-82:81-E3-BB"

At this point you should be able to access your ATA (GrandStream) device at the IP address you specified (from the machine you mapped it’s MAC address from). In my case, I’d point my web browser to: https://192.168.2.3/

Default Credentials
Admin Password admin
User Password 123

Login to the device with the Admin account (password is admin). The setup of this device can be a bit over-whelming, but bear with me; it’s worth it.

To start off with, you might want to click on the [Basic Settings] and configure your IP Address information. This will allow your device to be accessible from other PC’s in your house.

Now for the [FXS Port1] tab; the full settings can be found here. I’ll provide mine too should that be more helpful.
A point up front:

  • Your Primary SIP Server and Outbound Proxy should be set to the POP server your configured with your VOIP.ms account.

Below identifies the rest of the configuration; I realize that the configuration presented below can seem a bit overwhelming. But remember: you only have to do this part once and there are actually very little fields that require changing!

VOIP Solution - FXS Port1 - Page 1 of 5
VOIP Solution - FXS Port1 - Page 2 of 5
VOIP Solution - FXS Port1 - Page 3 of 5
VOIP Solution - FXS Port1 - Page 4 of 5
VOIP Solution - FXS Port1 - Page 5 of 5

Troubleshooting

Grandstream Expected LightsWhen you’ve set everything up, you should see the following lights on your Grandstream Device. Unless you see all 4 lights; you will ‘not’ be able to send/receive calls. Here are some hints if you don’t see the 4 lights identified:

  • No lights?: You might have a defective device. Double check that you’ve plugged in it.
  • No Internet light?: Check your IP settings on the [Basic Settings] from within the Grandstream configuration (webpage). If you’re plugged into a wireless router or your ISP’s modem, make sure you’ve selected dynamically assigned via DHCP.
  • No Link/Act Light?: You’re not able to communicate with Outbound Proxy or Primary SIP Server defined in the [FXS PORT1] from within the Grandstream configuration (webpage). Double check that you’ve got these right!
  • No Phone1 light?: The SIP User ID you specified could not be verified/validated by your VOIP provider. You’ll need to double check it. You may also need to make sure it’s activated too!.

Sources

  • VoIP.ms: The best VOIP solution I could find! Far cheaper and more feature rich than the alternatives (such as magicJack, Ooma, Vonage, etc…)!
  • VoIP.ms POP Server Listing: If you do decide to go with these guys, you’ll want to point your ATA device to a server close to you. This is the server listing you can choose from.
  • The National Do Not Call List (DNCL) (for Americans or Canadians: Even if you don’t plan on getting VOIP as described in this blog. You might want to at least add your cell phones to this list. It isn’t perfect, but it’s better than nothing!
  • GrandStream GS-HT702: This is the ATA device I personally use. This is also the device my blog focuses specifically.
  • GrandStream GS-HT702 VOIP Setup: Specifically the setup for VoIP.ms.
  • VoIP.ms Console: A fantastic Android App that lets you manage your VoIP.ms account from your mobile device (or tablet).
  • VoIP.ms SMS: Yet another fantastic Android App that lets you send and receive SMS (text) messages through your land line via your mobile device (or tablet).

Just for the record: VoIP.ms does not have a referral program (for me to benefit from), nor do I work for them at all. I speak highly of them because their pricing is incomparable to any of their competitors (in the best way possible), and their support is absolutely amazing. I’ve been using VoIP.ms for just over a year now and have been so happy with my service that I felt like blogging about it.

Awstats

AWStats Setup on CentOS

Introduction

AWStats is a great tool for gathering statistics about your website. It acquires everything it needs to know about your site strictly through your websites log files. AWStats is able to scan through these logs line by line and present them in a fantastic report. This report can really help you make strategic decisions going forward as well as spot any anomalies that might be taking place. The tool is smart enough to only scan newer log entries (from when it last ran) allowing you to run it again and again (as often as you want). Thus, once you set this tool up to run daily (or even hourly), you’ll have detailed statistics about your website you can call upon anytime.

AWStats collects information such as such as:AWStats Report

  • Who is visiting your site.
  • How many visitors you’re getting daily.
  • Where are they’re coming from (did a site link to you?)
  • Where is the visitor from (geographical location
  • … and on and on
  • The presentation of these collected statistics can be either via a website (HTML), XML and/or as a PDF file. The PDF is especially useful since it combines all of the multiple HTML pages (as presented) into one great big report with a table of contents and hyperlinks throughout it! The PDF is also really easy to navigate and pass along to others who might also be interested.

    Why Use AWStats over Google Analytics?

    Google Analytics Inaccuracy

    Google Analytics Inaccuracy

    The number one reason is because AWStats is much (,much) more accurate! AWStats also just works without ‘any’ changes to your website (literally – none at all). Google Analytics however requires you to add a small piece of JavaScript to every web page you want to track. Every time this tiny bit of JavaScript code executes, it passes the information along to Google. The problem is… if that little snippet of JavaScript doesn’t execute, then Google doesn’t track that user (and you’ll never know) because it just won’t get reported.

    It’s really easy to prevent this chunk of JavaScript from running too, you just have to have installed something like Ad-Blocker Plus, Disconnect and/or uBlock into your Web Browser (such as Firefox or Chrome). These plugins specifically block these tracking techniques and eliminate most (if not all) advertising the website might have too.

    It doesn’t mean that online analytic tools (like Google Analytics) are not good; no, not at all! But it’s just important to understand that they can’t (and truly aren’t) reporting everything that’s going on with your website and the traffic generated from it.

    Another point worth mentioning is that Google Analytics can not monitor and report statistics on traffic used by third party tools. Therefore you can’t use it to monitor any RESTful API services because the programs accessing it will never call these JavaScript snippets of code.

    It’s worth pointing out now that if you use AWStats, you’ll have the full picture! You’ll be able to easily identify any anomalies and detect certain forms of malicious intent! You’ll be able to monitor all of your internal (web based) services you may manage. From the public standpoint, you might be very surprised at how much more traffic your website is getting despite what online analytic tools will tell you!

    Let’s Get Started

    First you’ll want to install the proper packages. You should hook up to my repository and the EPEL repository as well! The EPEL repository hosts AWStats too, but mine is a newer version. We need the EPEL repository for it’s GeoIP packages since they get updated more often there:

    # CentOS 7 users can connect to EPEL this way:
    rpm -Uhi https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    
    # Similarly, you can hook up to my repository at http://nuxref.com
    # but here is a quick way of doing it (for CentOS/RedHat 7):
    rpm -Uhi http://repo.nuxref.com/centos/7/en/x86_64/custom/nuxref-release-1.0.0-4.el7.nuxref.noarch.rpm
    

    You should be good to go now; the following installs AWStats and a few extra tools to get the best out of it:

    # install awstats
    # install htmldoc too because it'll allow you to create a pdf
    # install geoip-geolite for the ability to track the IPs
    #        to countries
    # install perl-Geo-IP to look up the IP Addresses
    yum install awstats htmldoc geoip-geolite perl-Geo-IP
    
    

    AWStat In a Nutshell

    The steps below will require that you have set up the environment defined below. Obviously you’ll want to change these environment variables to suite your own needs:

    # First define our website as a variable.
    # We will use this value to track and store in an
    # organized structure.
    
    # Those who host other websites for people can
    # change this and virtually everything below will
    # and re-run everything to get stats for that too!
    WEBSITE=nuxref.com
    
    # AWSTATS Variable Data
    DATADIR=/var/lib/awstats/$WEBSITE
    
    

    Configuring AWStats: Step 1 of 3

    AWStats works from configuration files you create in /etc/awstats/. But it also needs a directory it can work within (we use /var/lib/awstats/). I’ve provided documentation around each line so you know what’s going on:

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    
    # First we need to setup our DATADIR; this is where
    # all our statistics and generated data will be placed
    # into:
    [ ! -d $DATADIR/static ] && \
        mkdir -p $DATADIR/static
    ln -snf /usr/share/awstats/wwwroot/icon \
       $DATADIR/static/icon
    ln -snf /usr/share/awstats/wwwroot/cgi-bin \
       $DATADIR/static/cgi-bin
    
    # Create a configuration file using our website
    # based on the awws.model.conf example file that
    # ships with AWStats
    sed -e "s|localhost\.localdomain|$WEBSITE|g" \
    	/etc/awstats/awstats.model.conf > \
    		/etc/awstats/awstats.$WEBSITE.conf
    
    ########################################
    # Now update our new configuration
    ########################################
    # Update the LogFile with our access.log file we'll
    # reference. This path doesn't exist yet but
    # we'll be creating it soon enough; leave this entry
    # untouched (don't change it to your real log path!):
    sed -i -e "s|^\(LogFile\)=.*$|\1=\"$DATADIR/access.log\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    # Disable DNS (for speed mostly)
    sed -i -e "s|^\(DNSLookup\)=.*$|\1=0|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    # For PDF Generation we need to update the relative
    # paths for the icons.
    sed -i -e "s|^\(DirIcons\)=.*$|\1=\"icon\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    sed -i -e "s|^\(DirCgi\)=.*$|\1=\"cgi-bin\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    

    Optionally Configuring GeoIP Updates

    The geolite data fetches us a great set of (meta) data we can reference when looking up IP Addresses (of people who visited our site) and determining what part of the world they came from. This information is fantastic when putting together statistics and web page traffic like AWStats does.

    First we want to configure AWStats to use the GEO IP Plugin:

    # Now configure our GEOIP Setup
    sed -i -e '/^LoadPlugin=.*/d' /etc/awstats/awstats.$WEBSITE.conf
    cat << _EOF >> /etc/awstats/awstats.$WEBSITE.conf
    LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"
    LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /usr/share/GeoIP/GeoIPCity.dat"
    _EOF
    

    Next we want to set up our GEO IP to update itself with the latest meta data for us automatically (so we don’t have to worry about it):

    # downloads all of the latest GEO IP content to
    # /usr/share/GeoIP with this simple command:
    geoipupdate
    
    # This IP information changes often; so the next
    # thing you want to do is create a cronjob to have
    # this tool fetch regular updates automatically for
    # us to keep the GEO IP Content fresh and up to date!
    cat << _EOF > /etc/cron.d/geoipdate
    0 12 * * 3 root /usr/bin/geoipupdate &>/dev/null
    _EOF
    

    Apache Users: Step 2a of 3

    AWStats depends on the log files to build it’s statistics from, so it’s important we point it to the right directory. Apache logs have been pretty much standardized and AWStats just works with them. If your web page is being hosted through Apache then your log files are most likely being placed in /var/log/httpd. If you’re using NginX (and not Apache), you can skip over this section and to Step 2b of 3 instead.

    Make sure AWStats knows it’s dealing with Apache log files (make sure you’ve still got the $WEBSITE variable defined from above):

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    ########################################
    # Apache Users Should run The Following
    ########################################
    # Now if you're logs are created from Apache you
    # need to run the following:
    # Log Format (Type 1 is for Apache)
    sed -i -e "s|^\(LogFormat\)=.*$|\1=1|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    
    

    Now what we want to do is take all of the logs files associated with our website in /var/log/httpd and build one great big (sorted) log file we can get all of our statistics out of:

    # logresolvmerge.pl is a fantastic tool that ships with
    # awstats and merges (and sorts) all of our logs. We
    # place the output into our $DATADIR (which we declared
    # earlier):
    /usr/share/awstats/tools/logresolvemerge.pl \
       /var/log/httpd/access.log \
       /var/log/httpd/access.log-????????.gz \
        > $DATADIR/access.log
    
    

    Nginx Users

    NginX logs have a slightly different format then the Apache logs and therefore require a slightly different configuration to work. If your web page is being hosted through NginX then your log files are most likely being placed in /var/log/nginx. If you’re using Apache (and not NginX), then you can skip over this section as long as you’ve already done Step 2a of 3 instead.

    Make sure AWStats knows it’s dealing with NginX log files otherwise it won’t be able to interpret them. Also be sure to have your $WEBSITE variable defined:

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    ########################################
    # NginX Users Should run The Following
    ########################################
    # If you're using NginX, you'll want to adjust
    # your awstat LogFormat entry as follows:
    sed -i -e "s|^\(LogFormat\)=.*$|\1=\"%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot\"|g" \
       /etc/awstats/awstats.$WEBSITE.conf
    

    Now we take all of the logs files associated with our website in /var/log/nginx and build one great big (sorted) log file we can get all of our statistics out of:

    # logresolvmerge.pl is a fantastic tool that ships with
    # awstats and merges (and sorts) all of our logs. We
    # place the output into our $DATADIR (which we declared
    # earlier):
    /usr/share/awstats/tools/logresolvemerge.pl \
       /var/log/nginx/access.log \
       /var/log/nginx/access.log-????????.gz \
        > $DATADIR/access.log
    

    Statistic Generation: Step 3 of 3

    At this point we have all the info we need

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    ########################################
    # (Create) and/or Update our Stats
    ########################################
    /usr/share/awstats/wwwroot/cgi-bin/awstats.pl \
       -config=$WEBSITE
    
    # The following builds us a PDF file containing all
    # of our statistics in addition to a website we can
    # optionally host if we want.
    # The following would allow you to gather statistics for
    # a given year:
    #   /usr/share/awstats/tools/awstats_buildstaticpages.pl \
    #      -config=$WEBSITE -buildpdf \
    #      -month=all -year=$(date +'%Y') \
    #      -dir=$DATADIR/static \
    #      -buildpdf=/usr/bin/htmldoc
    
    # This will build statistics with all the information we have:
    /usr/share/awstats/tools/awstats_buildstaticpages.pl \
       -config=$WEBSITE -buildpdf \
       -dir=$DATADIR/static \
       -buildpdf=/usr/bin/htmldoc
    
    # - The main website will appear as:
    #      $DATADIR/static/awstats.$WEBSITE.html
    #    But this 'main' website links to several other websites
    #    that can also all be found in the $DATADIR/static
    #    directory
    # - The pdf file will appear as:
    #      $DATADIR/static/awstats.$WEBSITE.pdf
    

    Consider throwing the above into a script file and having it ran in a cron job!

    Hosting The Statistics

    This option is purely optional; but but here is some simple configurations you can use if you want to access these generated statistics from your browser.

    Note: I intentionally keep things simple in this section. AWStats can be configured so that you can update your statistics via it’s very own website (see AllowToUpdateStatsFromBrowser directive in the site configuration). However I don’t recommend this option and therefore do not document it below.

    NginX

    A simple NginX configuration might look like this:

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    cat << _EOF > /etc/nginx/default.d/awstats.$WEBSITE.conf
       # Visit your statistics by browsing to:
       # if WEBSITE was equal nuxref.com, you'd visit the stats:
       # http://localhost/stats/nuxref.com/
       location /stats/$WEBSITE/ {
          alias   $DATADIR/$WEBSITE/static/;
          index  awstats.$WEBSITE.html;
    
          ## Set 1.2.3.4 to your own IP address and uncomment
          ## the entries below to 'only' allow yourself access to
          ## these stats:
          # allow 1.2.3.4/32;
          # deny all;
    
          location /stats/css/ {
              alias /usr/share/awstats/wwwroot/css/;
          }
    
          location /stats/icon/ {
              alias /usr/share/awstats/wwwroot/icon/;
          }
       }
    _EOF
    

    Don’t forget to reload NginX so it takes on your new configuration (and makes that statistics page visible):

    # Reload NginX
    systemctl reload nginx.service
    

    Apache

    # Make sure our environment variables are defined
    # WEBSITE and DATADIR
    cat << _EOF > /etc/httpd/conf.d/awstats.$WEBSITE.conf
       # Visit your statistics by browsing to:
       # if WEBSITE was equal nuxref.com, you'd visit the stats:
       # http://localhost/stats/nuxref.com/
       Alias /stats/$WEBSITE/ "$DATADIR/$WEBSITE/static/"
       <Directory "$DATADIR/$WEBSITE/static/">
          Options FollowSymLinks
          AllowOverride None
          Order allow,deny
          Allow from all
    
          ## Set 1.2.3.4 to your own IP address and uncomment
          ## the entries below to 'only' allow yourself access to
          ## these stats:
          # Order deny,allow
          # Deny from all
          # Allow from 1.2.3.4/255.255.255.255
       </Directory>
    _EOF
    

    Don’t forget to reload Apache so it takes on your new configuration (and makes that statistics page visible):

    # Reload Apache
    systemctl reload httpd.service
    

    Credit

    This blog took me a long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. I took the open source available to me and rebuilt it to make it an easier solution and decided to share it. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

    Sources

Nagios Ultimate Setup Part 3 of 3: NRPE

NRPE for Nagios Core on CentOS 7.x

Introduction

In continuation to part 1 and part 2 of this blog series… NRPE (Nagios Remote Plugin Executor) is yet another Client/Server plugin for Nagios (but can work with other applications too). Unlike NRDP, Nagios isn’t required for NRPE to function which means you can harness the power of this tool and apply it to many other applications too. It does however work ‘very’ well with Nagios and was originally designed for it.

If you read my blog on the NRDP protocol, then you’re already familiar with it’s push architecture where the Applications are responsible for reporting in their status. NRPE however works in the reverse fashion; NRPE requires you to pull the information from the Application Server instead. The status checking responsibility falls on the Nagios Server (instead of the Applications it monitors).

NRPE Overview

NRPE Overview


The diagram above illustrates how the paradigm works.
  • The A represents the Application Server. There is no limit to the number of these guys.
  • The N represents the Nagios Server. You’ll only ever need one Nagios server.

NRPE Query Overview

  1. The Nagios Server will make periodic status checks to the to the Application Server via the NRPE Client (check_nrpe).
  2. The Application Server will analyze the request it received and perform the status check (locally).
  3. When the check completes, it will pass the results back to the Nagios Server (via the same connection the NRPE Client started).
  4. Nagios will take the check_nrpe results and display it accordingly. If the check_nrpe tool can’t establish a connection to the NRPE Server (running on the Application Server), then it will report a failure.

    If the call to the check_nrpe tool takes to long to get a response (or a result) back, then it will be reported as a (timeout) failure. By default check_nrpe will wait up to a maximum of 10 seconds before it times out and gives up. You can change this if you find it too short (or to long).

Here is what you’re getting with this blog and the packaged rpms that go with it:

  • Nagios Core v4.x: updated RPM packaging which I continued to maintain and carry forward from my previous blogs.
    Note: You will need this installed in order to monitor applications with NRPE. See part 1 of this blog series for more information if you don’t already have it set up.
  • NRPE (Nagios Remote Plugin Executor) v3.x: My custom RPM packaging bringing NRPE v3.x to CentOS 7.x for the first time especially since I couldn’t find it available anywhere else (at the time of the this blog entry). I had to make a few modifications to it so that it would be easier used our environment such as:
    • I forwarded the useful patches from the old NRPE v2.x branch to the new NRPE 3.x branch that were applicable still.
    • I patched the SystemD startup script to work with CentOS/Red Hat systems.
    • A sudoer’s file is already to go for those who want to use sudo with their NRPE remote calls.
    • There is firewall configuration all ready to use with FirewallD.

    Best of all, with my RPMs, you can run SELinux in full Enforcing mode for that extra piece of mind from a security standpoint!

    The Goods

    For those who really don’t care and want to just jump right in with the product. Here you go!

    You can download the packages manually if you choose, or reference them using my repository:

    Package Download Description
    nrpe el7.rpm The NRPE Server: This should be installed on any server you want to monitor. The server will allow the machine to respond to requests sent to it via the check_nrpe (Nagios) plugin.
    nrpe-selinux el7.rpm An SELinux add-on package that allows the NRDP Server to operate under Enforcing Mode.

    Note: This RPM is not required by the NRDP server to run correctly.
    nagios-plugins-nrpe el7.rpm Our NRPE Client; this is the Nagios Plugin used to request information from the NRPE Server(s). You’ll only need to install this on the Nagios server (for the purpose of this blog). The RPM provides a small tool called check_nrpe that gets installed into the Nagios Plugin Directory (/usr/lib64/nagios/plugins).

    Through some simple configuration; Nagios can use the check_nrpe tool to monitoring anything you want just as long as the NRPE server is (installed and) running at the other end.

    Note: The source rpm can be obtained here which builds everything you see in the table above. It’s not required for the application to run, but might be useful for developers or those who want to inspect how I put the package together.

    NRPE Server Side Configuration

    The NRPE Server is usually installed onto all of the Application servers you want Nagios to monitor remotely. It provides a means of accepting requests to process (such as, what is your system load like?) and handling the request and returning the response back.

    Assuming you hooked up to my repository here, the NRDP server can can be easily installed with the following command:

    # Install NRPE (Server) on your Application Server
    yum install nrpe nrpe-selinux
    

    NRPE communicates through TCP port 5666; which means you may need to enable the ports in your firewall first to allow remote connections from Nagios. The below commands open up the protocol to everyone attached to your network. You should only perform this command if your Application Server will be running on a local private network:

    # Enable NRPE port (5666)
    firewall-cmd --permanent --add-service=nrpe
    firewall-cmd --add-service=nrpe
    

    If you’re opening this up to the internet, then you might want to just open the (NRPE) port exclusively for the Nagios Server. The following presumes you know the IP of the Nagios Server and will open access to ‘JUST’ that system:

    # Assuming 5.6.7.8 belongs to the Nagios Server you intend to
    # allow to monitor you; you might do the following:
    firewall-cmd --permanent --zone=public \
       --add-rich-rule='\
           rule family="ipv4" \
           source address="5.6.7.8/32" \
           port protocol="tcp" \
           port="5666" \
           accept'
    

    You’ll want to have a look at your NRPE Configuration as you will probably have to update a few lines. See /etc/nagios/nrpe.cfg and have a look for the following directives:

    Directive Details
    allowed_hosts This is added security for NRPE but can come back and haunt you if you ignore it (as nothing will work). You should specify the IP address of your Nagios server here and not allow anyone else! For example, if your Nagios server was 6.7.8.9, then you would put 6.7.8.9/32 here.
    dont_blame_nrpe This allows you to pass options into your remote checks. I personally think this is awesome, but there is no question that depending on the checks that accept arguments, it could could exploit content from your system you wouldn’t have otherwise wanted to share. This is specially the case if you grant NRPE Sudoers permission (discussed a bit lower). Set this value to one (1) to enable argument passing and zero (0) to disable it.
    allow_bash_command_substitution Bash substitution is something like $(hostname) (which might return something like node01.myserver). Set this value to one (1) to enable argument passing and zero (0) to disable it. I don’t use personally use this and therefore have it disabled on my system.

    Setting up NRPE Tokens

    Once your server is set up, there is one last thing you need to do. You need to associate tokens with some of the status checks you want to do. You see, NRPE won’t just execute any program you tell it to, it will only execute programs a specific way that you’ve allowed for. This is purely for security and it makes sense to do so! It’s really not that complicated, consider a token/command mapping like this (as an example):

    Token Command
    check_load # Check the system load:
    # – warning if greater than 10.0
    # – critical if grater then 20.0
    # – we map the check_load token to this command:
    /usr/lib64/nagios/plugins/check_load -w 10 -c 20

    You pass this information into NRPE by creating a .conf file in /etc/nrpe.d (assuming you’re using my RPMs) with the syntax:

    command[token]=/path/to/mapped/command

    So with respect to the example we started; it would look like this:

    command[check_load]=/usr/lib64/nagios/plugins/check_load -w 10 -c 20

    You can specify as many tokens as you like (each one has to be unique from the other). If you have a look in /etc/nrpd.d/, you’ll see one called common_checks.cfg which has a handful of useful commands to start you off with. You can add to this file or start another if you like (in the same directory with a .conf extension). Each time you make changes to the configuration (or another) you’ll need to signal NRPE to have it load any new changes you provided:

    # Restart our NRPE Service
    systemctl restart nrpe.service
    

    Here is a conceptual diagram that will help illustrate what was just explained here using the check_load example above:

    NRPE check_load Example

    NRPE check_load Example

    Sudoer’s Permission

    Substitute User Do (or sudo) allows you to run commands as other users. Most commonly people use sudo to elevate there permission to the root (superuser) privilege to execute a command. When the command has completed, they return back to their normal privileges.

    Some actions require you to have higher system authority to get anything good from them. This includes retrieve certain kinds of system/status information. For example, you can’t check how much mail is spooled and ready for remote delivery (on a Mail Server) unless you reduce the privileges of all of your stored mail (making it accessible by anyone); not a very good idea! But alternatively you can use sudo to grant one user permission to just run a command that can only fetch the number of mail items queued; this is a much better approach! Thus elevating a users permissions for just "special status checking only " commands isn’t so much of a risk. NRPE can be configured to have superuser permissions for it’s status checks which can allow you to monitor a lot more things! Here is how to do it:

    # First make sure that the sudoer's configuration doesn't
    # require tty (teletype terminals) endpoints to use. You
    # do so by commenting out the line that reads 'Defaults requiretty'
    # in the /etc/sudoers file (access it with the command visudo)
    # You can also do this with the below one-liner too
    sed -i -e 's/^\(Defaults[ \t]\+requiretty.*\)$/#\1/g' \
             /etc/sudoers
    
    # By doing this, you allow pseudo-teletype (pty) endpoints
    # to use the sudo command too. NRDP would be a pty service
    # for example since it's not an actual person running the
    # command.
    
    # Update our configuration to use the sudo command
    sed -i -e "s|^[ \t#]*\(command_prefix\)=.*|\1=/usr/bin/sudo|g" \
          /etc/nagios/nrpe.cfg
    
    # Restart our server if it's running
    systemctl restart nrpe.service
    

    SELinux users will want to also do this:

    # Allow NRPE/Nagios calls to run /usr/bin/sudo
    setsebool -P nagios_run_sudo on
    

    NRPE Client Side Configuration

    The client side simply consists of a small application (called check_nrpe) which connects remotely to any NRPE Server you tell it to and hands it a token for processing (provided your NRPE Server is configured correctly). The NRPE Server will take this token and execute a command that was associated with it and return the results back to you.

    # You'll want to be connected to my repositories for this to work:
    # See: http://nuxref.com/repo for more information
    # Install NRPE on the same server running Nagios
    yum install nagios-plugins-nrpe
    

    There is configuration already in place for you if you’re using my RPMS located in /etc/nagios/conf.d called nrpe.cfg. But you can feel free to create your own (or over-write it like so:

    cat << _EOF > /etc/nagios/conf.d/nrpe.cfg
    ; simple wrapper to check_nrpe for remote calls to NRPE servers
    define command{
       command_name check_nrpe
       command_line /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
    }
    
    ; check_nrpe with arguments enabled (enable the dont_blame_nrpe for this
    ; to work properly otherwise simply don't use it)
    define command{
       command_name check_nrpe_args
       command_line /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$
    }
    

    You can now send command tokens to servers running NRPE via Nagios. A Nagios configuration might look like this:

    cat << _EOF > /etc/nagios/conf.d/my.application.server01.cfg
    ; first we define a host that we want to monitor.
    ; If you already have a host configured; you can skip this part
    define host{
    ; Name of host template to use. This host definition will inherit
    ; all variables that are defined in (or inherited by) the
    ; linux-server host template definition. You can find this
    ; in /etc/nagios/objects/templates.cfg if you're interested
            use                     linux-server
    
    ; Now we define the server we're going to monitor
            host_name               my-application-server01.nuxref.com
            alias                   my-application-server01.nuxref.com
            address                 192.168.1.2
    
            statusmap_image         redhat.png
            icon_image              redhat.png
            icon_image_alt          CentOS 7.x
    }
    
    ; Now we want to define our monitoring service that talks to our
    ; Application server with NRPE installed on it:
    define service{
    ; Name of service template to use. This service definition will inherit
    ; all variables that are defined in (or inherited by) the
    ; local-service service template definition. You can find this
    ; in /etc/nagios/objects/templates.cfg if you're interested
    	use				local-service
            host_name                       my-application-server01.nuxref.com
    	service_description		Our System Load
    
    ; The Exclamation mark (!) lets nagios know that check_load is the argument
    ; we want to pass into our check_nrpe command we defined earlier (as $ARG1$)
    	check_command			check_nrpe!check_load
    }
    _EOF
    

    If you want to make sure it’s going to work, before you go any further you can run the same command you just told Nagios to do (above):

    # The syntax 'check_nrpe!check_load' gets translated to:
    #  /usr/lib64/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
    #
    # Which we can further translate (for testing purposes) to:
    /usr/lib64/nagios/plugins/check_nrpe -H 192.168.1.2 -c check_load
    

    If everything worked okay you should see something similar to the output:

    OK – load average: 0.00, 0.00, 0.00|load1=0.000;10.000;20.000;0; load5=0.000;10.000;20.000;0; load15=0.000;10.000;20.000;0

    You’ll want to reload Nagios to pick up on your new configuration so it can start calling this command too:

    # But before you reload it; it doesn't hurt to just check and make
    # sure your configuration is okay; You can test it out with:
    nagios -v /etc/nagios/nagios.cfg
    
    # correct any errors that display and rerun the above command until
    # everything checks out okay!
    
    # Now reload Nagios so it reads in it's new configuration:
    systemctl reload nagios.service
    

    NRPE vs. NRDP

    Which one should you use? Both NRDP and NRPE have their Pro’s and Cons.

    Benefits of using…
    NRPE NRDP
    You have central control over the checking periods of an application. You only ever need to open 1 TCP port; from a security standpoint; this is awesome!
    You don’t need Nagios to use this. If used properly, it can provide a great way to access remote servers for information and even execute administrative and maintenance commands. You only need to manage the Nagios configuration when a new server is added.
    Can send multiple status messages in one single (TCP) transaction.
    Can reflect a remote application status change immediately oppose to NRPE which one reflect the change until the next status check is performed.

    It should be worth noting that nothing is stopping you from using both NRDP and NRPE at the same time. You might choose an NRDP strategy for remote systems while choosing an NRPE strategy for all your systems residing in your private network. NRPE works over the internet as well, but just exercise caution and be sure to have SELinux running in Enforcing mode on all of the server end points.

    Credit

    This blog took me a very (,very) long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. All of the custom packaging described here was done by me personally. I took the open source available to me and rebuilt it to make it an easier solution and decided to share it. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

    Sources