Tag Archives: Security

Free Home Security Monitoring using Linux

Introduction

Home security is important to all of us but why should we pay someone to monitor our home when we can do it ourselves (for free)? I mean, imagine if it were possible to just receive a text message on your phone when your alarm has been set off. Imagine if you could check to see if you armed the alarm this morning before you left for work. If you had forgotten, wouldn’t it offer you a peace of mind to be able to arm it from wherever you are?

Some great reasons why one would choose to monitor their home themselves:

  1. Hundreds of Dollars in Yearly Savings: Think of how much money you invest in a 3rd party company that does nothing unless your alarm goes off. Even if you’re just paying $25 (before tax) a month for home security, we’re talking about a $300+ in yearly savings that you can pocket!
  2. Faster Emergency Response Time: Consider the scenario where a thief has broken into your home and has begun loading up his van with all of your valuables.

    Regardless of whether you’re paying for home monitoring or doing it yourself, the first thing that will happen is: Your alarm will sound within a 30+ seconds of the perpetrator’s entry; a very loud ear piercing siren I might add.

    • If you’re paying for a monitoring station service: They will call your home line to see if the threat is real or not (meanwhile your home is still being robbed). If they don’t receive an answer or get a busy signal, only then will they call the police. We’re easily a few minutes into the robbery before the authorities are even notified. The police will eventually arrive, they’ll file a report and you’re left to deal with insurance because the crime is long over and the criminals are long gone.
    • If you’re monitoring the home yourself: Well then your cell phone has already alerted you the second the alarm was set off. It’s a no brainer at this point, you can take action right then and there and call the police or 911 depending on if you’re home or not. You just immediately reduced the time the thief has to accomplish his crime in.
  3. Remote Control: Ensure the alarm is armed at any time. Change the alarm keycode remotely or disarm it from a distance to let a construction worker in. You can even arm/disarm your alarm from the comfort of your bed via your tablet, laptop or cell phone.

This blog focuses on the amazing hard work of Alarm Decoder and their team. It specifically focuses on their AD2USB device which allows you to connect a virtual keypad to your existing alarm system.

It’s also worth noting that I specifically focus on the Honeywell Vista 15p/20p models because that’s what I have. But I’m sure you could use the content of this blog and the other supported alarm systems Alarm Decoders AD2USB supports and follow along.

The Goal

The goal of this blog is to share with you my success, but more importantly:

  • To show you how easy it is to tap into an existing (pre-wired) alarm system without paying for a monitoring service.
  • Receive SMS (text) messages, emails or other notification services when your alarm is ever set off.
  • To be able to control the (security) keypad in your house from anywhere and at anytime.
  • To provide you all of this in a cookie cutter solution that works out of the box. I spent several hours packaging everything into self installing RPMs residing on my repository.

This blog will assume you’re both familiar with and have access to either CentOS (or Red Hat) 6.x or 7.x.

I’ll also assume you don’t have much experience with alarm systems (as I certainly didn’t at the start). I’ll try to save you as much research as I can and provide everything to you here. As a result, this blog is probably a bit longer then it needs to be since I get wordy trying to explain it.

Prerequisites

There are a few things you’ll need to be able to pull this feat off:

  1. Honeywell Vista 15p / 20pAn Ademco/Honeywell Vista 15p or 20p alarm system, but it doesn’t mean you can’t keep reading (just as long as your alarm panel is listed here as being supported).

    If you don’t have an alarm system pre-wired in your home already, it might be worth spending the money to have one installed (sensors and keypad included). It will be the last cost you make since the monitoring will be free from that point forward. If you don’t intend do this, then the this blog won’t be of much use to you.

    This blog assumes that the zones (sensors/alarms) are already pre-configured too. Thus, your sensors are all hooked up to the alarm system already.

    Note: If you’re currently paying a 3rd party company to monitor your home, then you MUST cancel your contract with them before you proceed any further!

  2. ad2usb DeviceYou’ll need full access to your Honeywell alarm system (usually in the basement) and at least one keypad (usually near the front entrance). Most alarm systems are (and should be) locked up in a small box accessible with a key. You will want this key because you’ll want to lock everything up when we’re done too!
  3. AD2USB Security Big PictureYou’ll need one NuTech AD2USB device which is what will bridge/connect your security system to your Linux Server. The AD2USB device effectively just becomes a second keypad attached to your alarm system.

    At the time this blog was written, the NuTech AD2USB devices cost about $88USD ($119.99 CAN) ; a one time fee. The cost is well worth it since it’ll pay itself off in 2-4 months (if you compare it to the costs of a 3rd party home monitoring solution).

  4. Mini USB Cable
    A Mini USB to USB Cable
    You’ll need one Linux server pre-configured with CentOS or Red Hat 6.x (or 7.x).
  5. Enough wiring to get from the locked up alarm system to your Linux server. In my case, I am hooked up just below the panel, so I just needed about 4 feet. You can order this cable with your purchase;it’s only about .50 cents a foot (very inexpensive).
  6. Wire strippers.
  7. A small (1/4″) Philips screw driver.
  8. A Mini-USB to USB cable (no longer then 10ft).

Alarm Installer Code

Before we can even look or use the NuTech AD2USB device, we need to first alert our current alarm system of it’s presence. To do this, you need to be able to re-program a tiny portion of the alarm system. Don’t worry, it’s not as complicated as it sounds! This step is done through the keypad already installed in your house (usually at one of your front entrance ways).

The installer code grants us unrestricted access to our entire alarm system (through it’s programming mode). The installer code is a 4 digit (numeric code) and won’t be the same one you’re used to keying in when activating/deactivate your alarm.

If you already know your installer code you can access the programming mode by keying in the following and skipping to the next section (substitute [ABCD] with the installer code):
[ABCD] + [8] + [00]

If you don’t know what your installer code is then don’t worry, just keep reading…

Again, I can’t stress enough that you make sure you are ‘NOT’ still hooked up with an external monitoring service at this point because these next steps will disconnect them from your home (and could cause them to treat the situation as a crime in progress):

  1. Unplug your alarm system’s main power by simply unplugging the transformer nearby from the wall.
  2. Disconnect the battery backup from our alarm system. No need to unplug both terminals, just unplug one for the time being. We’ll be restoring the power back soon enough.

    Note: At this point you’ve effectively powered off your alarm system completely.

  3. To gain access to the programming mode and ‘take back’ the alarm system for your own, you need to just plug the transformer back in (leave the battery unplugged now).You now have 30 seconds to locate your alarm keypad and press and hold both the asterisks [*] and the hash button [#] down at the same time (for about ~1 to 2 seconds).
  4. If the above step worked correctly for you, then you should see the value of 20 written on your keypad’s display. If not, then you’ll need to repeat all of the steps above again.

Note: If your battery backup for your alarm system is still unplugged, now would be a good time to just plug it back in (so we don’t forget about it later).

Keypad Configuration

This step requires you to be in-front of the same keypad described in the last section. More importantly, you must be in it’s programming mode. This means you either used your known installer key code or you booted the alarm up in such a away that granted you access to the programming mode (without a code). If you’re unsure what I’m talking about here, make sure to re-read the last section before proceeding!

Your alarm system is effectively organized as a series of registers (think of them as mailboxes organized by 2-3 digit addresses). Registers are basically a spot you can store and retrieve information from. You’ll always press the [*] followed by the register you want to modify (using it’s address), followed by the new value you want to store in it. Alternatively you can always press the hash [#] key followed by the register just to display it’s contents instead.

So with respect to the Mailbox analogy, programming our alarm system through our keypad works like so:

  • [*] + [mailbox] + [new value]
    Save content into a specific mailbox (by address) and then print it to the display.
  • [#] + [mailbox]
    Just print the content of the mailbox (by address) to the display.

Simple enough right?

Just to avoid any confusion, when I group numbers below as [20] and [1234], they can still be interpreted as pressing (sequentially / one after the other) [2] + [0] + [1] + [2] + [3] + [4]. It’s easier to group the actions separately for readability; this way you can think of it with respect to the mailbox/content analogy.

While in programming mode, execute the following commands:

Command Details
[*]+ [20] + [ABCD] Substitute ABCD with a 4 digit code you want to become the new programmer code. This is what you can use in the future to get into programming mode (without having to power off the entire system again).
[*]+ [41] + [*] Remove the primary phone #’s of any external monitoring station that may or may not be hooked up. Even if this register is already empty; it doesn’t hurt to run this again just to ensure it really is empty. We don’t want our alarm system calling out to anyone.
[*]+ [42] + [*] Remove the secondary phone #’s of any external monitoring station (if any is set). Again…even if this register is already empty; it doesn’t hurt to run this command again just to ensure it really is empty.. We don’t want our alarm system calling out to anyone.
[*] + [191] + [10] Enabled keypad slot at Address 18; this is what the AD2USB device defaults to using. This default address should work for everyone.
[*] + [99] Exit the installer mode returning the keypad to it’s regular state

We’re not done yet though, if this is you first time using your alarm system, it might be wise to set a new Master Code and clear all existing User Codes. In case you didn’t already know, there are actually 3 types of users that are associated with our alarm system:

  1. The Installer: A key code that allows us to add/remove new keypads, acknowledge new sensors, configure zones, etc. In our case, our alarm system will only allows for 1 designated Installer Code.
  2. The Master: A key code that allows us to to create/delete Users of the system. In our case, our alarm system only allows us to have 1 designated Master Code.
  3. The User: Our alarm system allows us to program up to 31 of them (32 if you count the master). Residential owners will probably only need to set one user while a commercial business owner might set several for each of their employees. It will be our User Code that we’ll be using when interfacing with our alarm system 99% of the time. This blog will focus on just dealing with 1 main user.

So if you inherited this alarm, you may not know what was previously configured into it. You ideally should set a new Master Code (different then the Installer Code) and make sure there are no lingering User Codes active that we don’t know about.
Note: The commands identified above are called outside of programming mode. They can be issued at any time as long as your alarm is not armed. If you’re still in programming mode, just type [*] + [99] to exit it.

Command Details
[Installer Code] + [8] + [02] + [DEFG] Substitute [DEFG] with your new 4 digit Master Code using the Installer Code we setup above.
[Master Code] + [8] + [03] + [WXYZ] Substitute [WXYZ] with your new 4 digit User Code using the Master Code. This will become our main User account; the [03] identifies us as user #1. This is a code you’ll want to keep secret but can safely tell those whom you trust with access to and from your home.
[Master Code] + [8] + [04] + [#] + [0]

[Master Code] + [8] + [05] + [#] + [0]

[Master Code] + [8] + [06] + [#] + [0]

[Master Code] + [8] + [33] + [#] + [0]

Delete User #2 ([04]), #3 ([05]) and , #4 ([06]) in our alarm system (in case they exists). This step is entirely optional, but worth doing since you really don’t know how many users are actually set up from whomever had it before you.

If you’re really paranoid, you’ll iterate through all of the combinations identified here to cover of all 31 possible users where the last user (#31) is identified by [33]. In most cases your system will probably have just a few users (worst case) in it. Thus it’s probably safe to stop at [06] (user #4). I’ll let you use your own paranoia and discretion here as to how far you go.

You now have a security system and 3 Codes. The Installer and Master codes are useful to keep documented (but hidden) somewhere. The User Code will be the main code we use from now on; this code is safe to share with those you share your home with; obviously keep it safe from others.

Hook Up our NuTech AD2USB Device

First of all; let me point out to you that the Alarm Decoder’s Installation videos are a much better reference then anything I’ll identify in this section. I strongly encourage you to check the videos out (they’re only a few minutes) and get properly set up.

But in short, it really just boils down to connecting the AD2USB Device to our Home Alarm System and getting the wires correctly matched up and fastened tightly. From there you just connect the AD2USB device to your Linux Server via the mini-usb cable.

NuTech AD2USB Wiring In a Nutshell
NuTech AD2USB Wiring In a Nutshell

Your goal is to see the little green light blink; this is the devices internal heartbeat. It’s a good thing when this starts to flash on and off; if it isn’t, you’ll want to revisit your wiring!

Here is what my AD2USB device looked like after it was all hooked up. Obviously I can’t capture a pulsating green light in a single photo, so you’ll just to have to take my word for it that the light was blinking on and off. 🙂

NuTech AD2USB Device
NuTech AD2USB Device

At the end of the day, the AD2USB device is nothing but ‘another’ alarm keypad in your house. As far as the Alarm System is concerned, there is no distinguishable difference at all. What makes the device so powerful is the fact that now we can interface with it using software from anywhere!

Software Installation

Now there are many ways to go about installing the software; specifically our core components: alarmdecoder, ser2sock, and alarmdecoder-webapp. One way is to use the instructions right on their GitHub page. If you go this route, then you’re done with this section of the blog.

However, I’ve spent hours (sadly) making it possible for these tools to just work in both CentOS 6.x and 7.x via installable RPMs. I strongly encourage you to go this route for several reasons:

  • You won’t be required to compile anything.
  • I handle SELinux for you so you won’t have to to disable it!
  • I handle all of the initial (and complicated configuration) out of the box for you.
  • I handled all of the system dependencies for you.
  • It’s easier to provide safe upgrades and version control through RPMs. Relying on pypi (pip) instead will make it difficult to reproduce your working setup since the packages there change constantly (for better of for worse).

If you want the RPM approach, then keep reading…

Install Core Components

You must be connected to my repository for this to work:

# Assuming you're correctly hooked up to the nuxref repositories:
yum --enablerepo=nuxref --enablerepo=nuxref-shared \
    alarmdecoder ser2sock \
    alarmdecoder-webapp alarmdecoder-webapp-selinux

Note: the SElinux package has to interact with the kernel to enable 4 web ports Alarm Decoder (WebApp) uses. During this time, it will appear as though your installation froze and/or hung. Do not abort this task, it’s completely normal to take up to a minute or more to do this. If you’re not using SELinux then you don’t need this package.

Enable ser2sock; this will convert your AD2USB device (in /dev/) into a TCP/IP stream. This is much easier to work with (and debug)

# Enable ser2sock and start it
chkconfig ser2sock on
service ser2sock start

If your ser2sock doesn’t start, then you have a problem with your AD2USB configuration and/or hookup. Feel free to have a look at /etc/ser2sock/ser2sock.conf to see the configuration it’s using. Alternatively, the next section discusses some troubleshooting steps you can do.

Start up our Alarm Decoder Webapp

Even if you’re using systemd, these commands will work (it’ll call the proper command for you).

# Enable gunicorn (back-end webserver)
chkconfig ad2web on
service ad2web start

# Enable NginX (front-end web hosting and static file control)
chkconfig nginx on
service nginx start

# If it was already running, then:
service nginx reload

You’ll need to open ports 5080 and 5443 on your firewall. CentOS/Red Hat 6.x users should be able to apply something like this in their /etc/sysconfig/iptables file:

# add (something like) this to your /etc/sysconfig/iptables
#---------------------------------------------------------------
# Home Security Monitoring
#---------------------------------------------------------------
# 192.168.1.0/24 is the subnet of my local home network and eth0
# is the device it access.
-A INPUT -i eth0 -s 192.168.1.0/24  -p tcp --dport 5080 -j ACCEPT
-A INPUT -i eth0 -s 192.168.1.0/24  -p tcp --dport 5443 -j ACCEPT

If you’re using CentOS 7.x, then you’ll need to use firewalld to accomplish the same task:

# Open the port in your home network
firewall-cmd --zone=home --add-port=5080/tcp --permanent
firewall-cmd --zone=home --add-port=5443/tcp --permanent
firewall-cmd --reload

I do not recommend opening the ports to the public (zone) network unless you strap on a bit of extra security such as fail2ban. This blog is already long enough, so I won’t discuss that here. Although, you can check out a blog I did on security that can get you started with fail2ban (6.x users).

It’s worth confirming at this point that everything is up and running too:

# This command just checks that we're properly listening on the
# ports we care about (with respect to Alarm Decoder):
netstat -nat | egrep LISTEN | egrep ':(10000|5(00[01]|080|443))'

The command above should produce (or similar to):

tcp        0      0 0.0.0.0:5443                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:5000                0.0.0.0:*                   LISTEN
tcp        0      0 127.0.0.0:5001              0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:10000               0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:5080                0.0.0.0:*                   LISTEN

The TCP ports break down as follows:

  • 5080: NginX non-encrypted hosting of the Alarm Decoder WeAapp.
  • 5443: NginX secure hosting of the Alarm Decoder WeAapp.
  • 10000: Ser2Sock (this is converting you’re AD2USB to a TCP/IP stream that the Alarm Decoder WebApp can read and display to you).
  • 5000: This is forked from the gunicorn instance to handle all incoming requests from NginX
  • 5001: This is the gunicorn application which helps with handling some of the backend tasks such as the detection of your AD2USB device.

You should be able to access the keypad website now; simply visit http://localhost:5080/ or it’s secure version https://localhost:5443/. If you’re configuring this on another server, we’ll then you’ll need to access the host/ip accordingly.

Alarm Decoder WebApp Installation
Alarm Decoder WebApp Installation

You can accept the defaults for everything, just keep pressing next.

Next 3 Setup Screens
Next 3 Setup Screens

When you’re all done with the setup and have created your administrative account, you’ll be able to sign in an access your new virtual keypad.

Alarm Decoder Keypad
Alarm Decoder Keypad

You can explore the software and take advantage of all of it’s features. You can setup a twilio account and have the Alarm Decoder WebApp send you a text message whenever an even occurs. You can have it email you, and/or use a variety of other messaging services too!

If the page still isn’t coming up, it might be because you have SELinux enabled. Unlike other bloggers, I will NOT tell you to disable it (although that’s an option). Instead I’d advise that you install the SELinux package I put together:

yum --enablerepo=nuxref alarmdecoder-webapp-selinux

Some other useful things worth knowing:

  • /etc/nginx/conf.d/alarmdecoder-webapp.conf: Here is your NginX configuration file. This is the front end of your keypad.
  • /etc/alarmdecoder-webapp/alembic.ini: This is another configuration file used by the website. It defines information such as the database type and location. By default, it’s just a simple sqlite database. But feel free to point it to a PostgreSQL or MySQL database too (see the sqlalchemy.url directive).
  • /etc/ser2sock/ser2sock.conf: This is ser2sock configuration file.
  • /etc/alarmdecoder-webapp/ad2web.conf.py: This is where your gunicorn configuration is defined.
  • /var/lib/alarmdecoder-webapp/instance/db.sqlite: Here is the database the website uses and interfaces with.
  • /var/lib/alarmdecoder-webapp/instance/logs/: Here are where you’re logs will appear from the webapp.

Troubleshooting

  1. First make sure CentOS (or Red Hat) can see your NuTech AD2USB device properly.
    # This ideally should return something like /dev/ttyUSB0
    find /dev -mindepth 1 -maxdepth 1 -type c -name 'ttyUSB*'
    
    # If you're using my RPM files, you'll also have another
    # entry you can look for. The following should output 
    # something like /dev/alarm-XXXXXXXX where the X's represent
    # the Serial # of your NuTech AD2USB
    find /dev -mindepth 1 -maxdepth 1 -type l -name 'alarm-*'
    

    If you can’t find your device, then:

    • Double check that the USB connections are firmly plugged in to both your PC and the AD2USB device.
    • You’ll want to see a pulsating green light on the AD2USB device signifying it’s getting power and running.
    • NuTech also make mention that the USB cable you use can not be longer then 10ft. Ideally the shorter the cable the better though (less noise/interference).

    If your device does exist, then read on to the next step…

  2. Ensure that the Honeywell Alarm System setup has been correctly configured to support a second keypad. This was discussed above in the Keypad Configuration section of the blog. Specifically you need to be in Programming Mode and enable keypad address 18 (done via the command: [*] + [191] + [10]).

    You can verify that you’ve configured this correctly by accessing the Programming mode of your alarm keypad and typing: [#] + [191]. The device should output either a 10 or 0A (both are valid responses).

  3. Make sure your NuTech AD2USB can properly communicate with your Alarm system.
    I took extra care and time packaging everything, so I actually have another RPM you can install that may help you out:

    # Assuming you're correctly hooked up to the nuxref repositories:
    yum --enablerepo=nuxref alarmdecoder-cli
    

    This will grant you access to a few scripts I created based on a couple of great examples the Alarm Decoder team put on their GitHub page:

    # These commands require you to be the superuser (root)
    # or a user that is as a member of the 'alarm' group.
    
    # Test your data feed (through ser2sock):
    ad2-disp-sock
    
    # Or test your block device directly:
    ad2-disp-dev
    

    Note: If ser2sock is running properly, you will not be able to test the block device (using ad2-disp-dev) until you stop it first. So don’t panic if the ad2-disp-sock works and ad2-disp-dev doesn’t!

    You can adjust these test tools by editing /etc/ad2cli.conf (if the defaults
    aren’t working for you).

    If you can’t communicate properly, then you may need to recheck your wiring. Review the NuTech YouTube videos again if you need to.

    If you still can’t communicate with your alarm system after confirming all of the suggestions above, then you may have a faulty device.

Don’t give up if you’re having problems. It’s worth searching the Alarm Decoder Forums for an answer. Post a question there too if you like. The Alarm Decoder team are a great bunch of people always willing to help.

What If

  • What if the thief cuts the power?
    In my case, my alarm system (yours should too) has a battery backup locked up tight with it. So it’ll keep monitoring in this situation no matter what. It wouldn’t hurt to connect both your server internet modem into a Uninterruptible Power Supply (UPS). These things are relatively cheap for what they offer. A UPS would certainly handle this situation.
  • What if the thief powers off the computer within the 30 second deactivation window?
    Well, if they are this clever then yes, they cut you off from remote monitoring. But that’s all they do. Keep in mind that the 30 second deactivation timer is still counting down. It will reach zero… and when it does…

    A properly installed alarm system will come with a siren (I know mine does) and they are quite loud and irritating! The siren will sound regardless of the NuTech Alarm Decoder, Internet or PC get shut off or unplugged. It might be a good idea to make sure your siren is installed in the attic where it is difficult to get to (and disable). Most people and installations place these devices in the basement. This is silly because a thief can just smash it or yank the wire from it to kill the sound. Either way, unless the thief knows your setup, he’s certainly playing with fire if he thinks silencing the siren will keep him safe.

    Consider a backup plan; I mean what would the alarm company do if they could no longer suddenly monitor you’re home. They’d probably immediately attempt to contact you and then call the police. You could do the same. Call a neighbor, call a friend. The fact you even just have a heads up on the situation is more then the guy who doesn’t have any monitoring solution at all.

    Another great tool you can use is Nagios; I wrote a blog on it a while ago. Pair that up with aNag (for Google Devices) or iNag (for Apple Devices), and you’ll be monitoring your ability to monitor. You can be notified immediately if your monitoring capabilities diminish.

    On a side note, consider notifying your (trustworthy) neighbor(s) too of weeks you intend to be out of town, I’m sure they can alert authorities for you if they hear your house alarm going off.

Home Security Tips

Remember, since you’re doing the monitoring yourself, you need to be prepared for when or if a situation ever arises. There is no longer a remote monitoring station looking out for you. But remember that even they can fail to pull through, so the following information is useful regardless of what your monitoring situation is.

I am not a trained professional when it comes the safety and responsibilities (nor will I claim to be). But there are clear cut obvious things worth stating up front that you must always consider:

  • If you’re at home at night and the alarm goes off:
    1. It’s our human nature to panic, but just don’t let the panic take over the situation.
    2. Don’t ever second guess the potential danger of the situation!
    3. Do not ever decide to be brave and investigate the situation unless you’re absolutely sure!!
    4. Call 911 immediately. If the phone line is dead, then use a cell phone if you have one.
  • If you’re at work or on vacation and the alarm goes off:
    1. Ask Yourself? Could it be your spouse? A Child? A Pet? Does the timing of this alarm jive with a notable time of day such as after school (maybe the kids got home)? Perhaps it’s around the time your spouse returns from work or other family member?
    2. Can you contact a specific person in mind directly based on the time of day (with respect to the alarm going off)?

      Remember though: if this truly is a robbery, then time is not on your side! The longer you wait, the more your home is exposed.

    3. If you can’t be sure, then call the local police immediately!. If you fear someone else could be home during this time and at risk then call 911! Don’t think twice!

Make sure you have your local police contact information programmed in your cell phone. Alternatively, keep it on a sheet of paper in your wallet or purse.

If you have children, make sure they know what to do if they hear the alarm go off. If they have access to a phone, they should know how to call 911 during these kind of emergencies. Obviously their safety becomes your number one concern, so be sure to educate them to hide someplace close by. Having them leaving the room their in when the alarm is going off could put them in serious jeopardy depending on the situation.

Credit

This blog took me a very (,very) long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Special thanks to the Alarm Decoder team; their support made it possible for me to write this blog. In return I was able to make it easier for everyone else!

Sources

Block and Unblock Ports on Your Firewall Quickly

Introduction

This blog just shares and explains the function of a script I created for *nux based systems that can be used to quickly block and unblock ports on your firewall. It works with both IPTables and FirewallD. It can also be easily configured to work with a cron and ran at key times.

There are several reasons why you might want to quickly block a port on your firewall.

  • Maybe you suspect someone is intruding or violating your system.
  • Maybe you want to leave your computer running (it’s crunching data for you), but for security reasons you want to close all of the ports when you’re not around (such as after work hours or over the weekend).

The Goods

At the present time, the script supports blocking inbound tcp and udp ports. It also supports blocking port ranges.
Simply copy this into a script file which I called blockport.sh (don’t forget to make it executable) and place it in your path:

#!/bin/sh
# Name: blockport.sh
# Author: Chris Caron <lead2gold@gmail.com>
# Date: Jul 17th, 2015
# Description: A simple script that blocks and unblocks outbound traffic
#              The intent is to set up a cron that will block stuff at night
#              and then unblock it in the morning every day preventing
#              any unnecessary things from making outbound requests.
#
# Dependencies: sed, logger, grep, and iptables
#
# This script must run as root!

SCRIPTNAME=$(basename $0)
IPTABLES_BIN=iptables
IPTABLES_PRINT=echo

# Set the command to what you want to actually do
# some scripts explicitly reference the above
IPTABLES=$IPTABLES_PRINT
IPTABLES=$IPTABLES_BIN

# Default Line Numbers (where inserts will take place
IPTABLES_OUTPUT_LINE=${IPTABLES_OUTPUT_LINE:=1}
IPTABLES_INPUT_LINE=${IPTABLES_INPUT_LINE:=1}

show_syntax () {
   echo "Syntax: \$> $SCRIPTNAME <Action> [Options]"
   echo "  Actions:"
   echo "     -l"
   echo "     --list                  List the ports blocked by this script."
   echo
   echo "     -bo PORT"
   echo "     --block-outbound=PORT   Blocks outbound ports (both udp and tcp). You can"
   echo "                              use space or comma to delimite multiple ports"
   echo "     -uo PORT"
   echo "     --unblock-outbound=PORT Unblocks outbound ports (both udp and tcp)"
   echo "                              previously blocked. You can use space or comma to"
   echo "                              delimit multiple ports"
   echo "     -bi PORT"
   echo "     --block-inbound=PORT    Blocks inbound ports (both udp and tcp). You can"
   echo "                              use space or comma to delimite multiple ports"
   echo "     -ui PORT"
   echo "     --unblock-inbound=PORT  Unblocks inbound ports (both udp and tcp)"
   echo "                              previously blocked. You can use space or comma to"
   echo "                              delimit multiple ports"
   echo ""
}

clean_ports(){
   # A Simple function that cleans up port ranges
   local PORTS=$(echo $1 | sed -e 's/[, \\/]\+/ /g' -e 's/[;-]\+/:/' \
             -e 's/[^:0-9 ]//g' -e 's/^[^0-9]\+//g' -e 's/[^0-9]\+$//g')
   [ -z "$PORTS" ] && return 1
   echo "$PORTS"
   return 0
}

indexes() {
   # Possible types are OUTPUT, INPUT, and FORWARD
   local TYPE=$1
   # A simple function that returns the index(es) of a iptable
   # entry (if it's blocked or not).
   local PORTS=$(clean_ports $2)
   [ $? -ne 0 ] && return 1

   local INDEXES=""
   # This magical line was constructed off of what i learned here:
   # http://unix.stackexchange.com/questions/129087/grep-log-and-get-text-between-log-delimiters
   # It extracts the DROP lines we created in the OUTPUT Chain
   for PORT in $PORTS; do
      INDEX=$($IPTABLES_BIN -nL --line-numbers | \
         grep -a -zPo "(\n?Chain $TYPE [^\n]*\n)\K(.|\n)+?[^\n][^Cc](.|\n)+?(?=\nChain [^\n]*\n)" | \
                egrep -a 'DROP' | egrep '^[0-9]' | egrep " dpts?:$PORT[ \t]*\$" | \
                sed -e 's/^[ \t]*\([0-9]\+\)[^0-9].*/\1/g')
      INDEXES="$INDEXES $INDEX"
   done

   [ -z "$INDEXES" ] && return 1

   # Sort the INDEXES (largest # to smallest) because we want to
   # process the list backwards
   INDEXES=$(echo $INDEXES | tr -s '[:space:]' '\n' | sort -n -r | uniq)
   echo $INDEXES
   return 0
}

unblock() {
   # Possible types are OUTPUT, INPUT, and FORWARD
   local TYPE=$1
   # A simple function that returns the index(es) of a iptable
   # entry (if it's blocked or not).
   local PORTS=$2

   # Defaults
   [ -z "$TYPE" ] && TYPE=INPUT

   # Stores the indexes (if set)
   local INDEXES="$(indexes $TYPE $PORTS)"
   [ -z "$INDEXES" ] && return 0

   # Sort the INDEXES (largest # to smallest) because we want to
   # process the list backwards
   INDEXES=$(echo $INDEXES | tr -s '[:space:]' '\n' | sort -n -r | uniq)

   for INDEX in $INDEXES; do
      $IPTABLES -D $TYPE $INDEX
   done
   logger "INFO - blockport.sh: Unblocked $TYPE $PORTS"
   return 0
}

block(){
   # Possible types are OUTPUT, INPUT, and FORWARD
   local TYPE=$1
   # A simple function that returns the index(es) of a iptable
   # entry (if it's blocked or not).
   local PORTS=$(clean_ports $2)
   [ $? -ne 0 ] && return 1

   # Defaults
   [ -z "$TYPE" ] && TYPE=INPUT

   # If indexes already exist, then we don't have to do anything
   local INDEXES="$(indexes $TYPE $PORTS)"
   [ ! -z "$INDEXES" ] && return 0

   local LINE=$(eval "echo \$IPTABLES_${TYPE}_LINE")
   [ -z $LINE ] && LINE=1
   for PORT in $PORTS; do
      $IPTABLES -I $TYPE $LINE -p tcp -s 0/0 --destination-port $PORT -j DROP
      $IPTABLES -I $TYPE $LINE -p udp -s 0/0 --destination-port $PORT -j DROP
   done
   logger "INFO - blockport.sh: Blocked $TYPE $PORTS"
   return 0
}

list() {
   echo
   for TYPE in "INPUT" "OUTPUT" ; do
      echo "Listing $TYPE Ports Blocked:"
      $IPTABLES_BIN -nL --line-numbers | \
         grep -a -zPo "(\n?Chain $TYPE [^\n]*\n)\K(.|\n)+?[^\n][^Cc](.|\n)+?(?=\nChain [^\n]*\n)" | \
         egrep -a 'DROP' | egrep '^[0-9]' | egrep " dpts?:[0-9:]+[ \t]*\$"
      echo
   done
   return 0
}

##########################################################
##                          Main                        ##
##########################################################
PATH=/bin:/sbin:/usr/bin:/usr/sbin
if [ $(whoami) != "root" ]; then
   echo "Error: you must be root to execute this script."
fi

ACTION="x"
RETVAL=0
while : ; do
   case $1 in
      -l) list; exit 0 ;;
      --list) list; exit 0 ;;
      -bo) ACTION='bo';
           block "OUTPUT" $2;
           [ $? -ne 0 ] && RETVAL=1
           shift; shift ;;
      --block-outbound) ACTION='bo';
           block "OUTPUT" $(echo $1 | sed -e 's/--block-outbound=//g' -e "s/'//g" -e 's/\"//g')
           [ $? -ne 0 ] && RETVAL=1
           shift ;;
      -uo) ACTION='uo';
           unblock "OUTPUT" $2;
           [ $? -ne 0 ] && RETVAL=1
           shift; shift ;;
      --unblock-outbound) ACTION='uo';
           unblock "OUTPUT" $(echo $1 | sed -e 's/--unblock-outbound=//g' -e "s/'//g" -e 's/\"//g')
           [ $? -ne 0 ] && RETVAL=1
           shift ;;
      -bi) ACTION='bi';
           block "INPUT" $2;
           [ $? -ne 0 ] && RETVAL=1
           shift; shift ;;
      --block-inbound) ACTION='bi';
           block "INPUT" $(echo $1 | sed -e 's/--block-inbound=//g' -e "s/'//g" -e 's/\"//g')
           [ $? -ne 0 ] && RETVAL=1
           shift ;;
      -ui) ACTION='ui';
           unblock "INPUT" $2;
           [ $? -ne 0 ] && RETVAL=1
           shift; shift ;;
      --unblock-inbound) ACTION='ui';
           unblock "INPUT" $(echo $1 | sed -e 's/--unblock-inbound=//g' -e "s/'//g" -e 's/\"//g')
           [ $? -ne 0 ] && RETVAL=1
           shift ;;
      -h) show_syntax ; exit 0 ;;
      --help) show_syntax ; exit 0 ;;
       *) if [ -z "$1" ]; then break; fi
          echo "[error] Invalid option '$1' specified; see --help (-h) for more info."
          exit 1
          ;;
   esac
done

if [ $ACTION == "x" ]; then
   show_syntax
   exit 1
fi

exit $RETVAL

Again, I state: this script must be ran as root (because it wraps IPTables). So stick sudo in front of it’s calls if running as a regular user (assuming you’re set up with sudoer’s privileges).

Syntax

Syntax: $> blockport.sh  [Options]
  Actions:
     -l
     --list                  List the ports blocked by this script.

     -bo PORT
     --block-outbound=PORT   Blocks outbound ports (both udp and tcp). You can
                              use space or comma to delimit multiple ports
     -uo PORT
     --unblock-outbound=PORT Unblocks outbound ports (both udp and tcp)
                              previously blocked. You can use space or comma to
                              delimit multiple ports
     -bi PORT
     --block-inbound=PORT    Blocks inbound ports (both udp and tcp). You can
                              use space or comma to delimit multiple ports
     -ui PORT
     --unblock-inbound=PORT  Unblocks inbound ports (both udp and tcp)
                              previously blocked. You can use space or comma to
                              delimit multiple ports

Demo

Here is a simple example that just blocks 2 ports. We can chain more than one port by placing a comma in between each one. It is also valid to keep using one switch after another (they’ll be executed in order).

# Here is how easy it is to use; first we'll block (inbound) ports 80 and 443
blockport.sh -bi 80,443

# This is also valid syntax:
#    blockport.sh -bi 80 -bi 443

# We know they're blocked now, but we can have a look anyway:
blockport.sh -l
# The output will look like this:
# Listing INPUT Ports Blocked:
# 1    DROP   udp  --  0.0.0.0/0   0.0.0.0/0   udp dpt:443
# 2    DROP   tcp  --  0.0.0.0/0   0.0.0.0/0   tcp dpt:443
# 3    DROP   udp  --  0.0.0.0/0   0.0.0.0/0   udp dpt:80
# 4    DROP   tcp  --  0.0.0.0/0   0.0.0.0/0   tcp dpt:80
#
# Listing OUTPUT Ports Blocked:
#

# We can reverse this by typing:
blockport.sh -ui 80,443

# This is also valid syntax:
# blockport.sh -ui 80 -ui 443

You can do ranges too; just use the colon (:) or hyphen (-). In the example below, we block a range of outgoing traffic from a system:

# Below blocks outbound ports 20, 21, and all ports (and including) 8000-8500.
blockport.sh -bo 20,21,8000-8500

# This is also valid syntax:
#    blockport.sh -bo 20 -bo 21 -bo 8000-8500

# We know they're blocked now, but we can have a look anyway:
blockport.sh -l
# The output will look like this:
# Listing INPUT Ports Blocked:
#
# Listing OUTPUT Ports Blocked:
# 1    DROP   udp  --  0.0.0.0/0   0.0.0.0/0   udp dpts:8000:8500
# 2    DROP   tcp  --  0.0.0.0/0   0.0.0.0/0   tcp dpts:8000:8500
# 3    DROP   udp  --  0.0.0.0/0   0.0.0.0/0   udp dpt:21
# 4    DROP   tcp  --  0.0.0.0/0   0.0.0.0/0   tcp dpt:21
# 5    DROP   udp  --  0.0.0.0/0   0.0.0.0/0   udp dpt:20
# 6    DROP   tcp  --  0.0.0.0/0   0.0.0.0/0   tcp dpt:20

# We can reverse this by typing:
blockport.sh -uo 20,21,8000-8500

# This is also valid syntax:
# blockport.sh -uo 20 -uo 21 -uo 8000-8500

Note: You can only unblock what you block. Here is an example of what I mean:

# Blocking a range:
blockport.sh -bo 8000-8500

# You CAN NOT just unblock a port from it (this will not work):
blockport.sh -uo 8400

# Similarly, you can not block individual ports and then try to unblock them
# as a range:
blockport.sh -bo 20,21

# You CAN NOT just unblock this as a range (this will not work):
blockport.sh -uo 20-21

Caution
This script is intended to be an instant port blocker. It intentionally destroys any pre-established connections utilizing the port marked. Keep this in mind so that you don’t block yourself out of your own Server or VPS. Hence DON’T CLOSE PORT 22 unless you know what and why you’re doing it. You have been warned! 🙂

Use Cases
Suppose you want to deny access out of a server you host for your company after hours, you could create a cron like this:

# Block defined outbound ports at around 5:30pm every evening on Weekdays
# (Mon - Fri)
30 17 * * 1-5 /root/bin/blockport.sh -bo 80,443,22,21 &>/dev/null

# Unblock the defined ports every morning at 7am on Weekdays
# (Mon - Fri) keeping them blocked over the weekends
0 7 * * 1-5 /root/bin/blockport.sh -uo 80,443,22,20,21 &>/dev/null

Alternatively, maybe something looked bad in /var/log/messages or /var/log/audit/audit.log to you and you simply just want to immediately block the port.

Credit

If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Permanently Ban Those Caught By Fail2Ban

Introduction

Fail2ban is probably one of the best intrusive detection based tools an administrator can deploy onto their system. This is especially the case if your system is connected to the internet. If you aren’t already using it; consider reading my blog entry here that talks about it.

In this blog, I provide a scripted solution that will capture the current list of banned users from Fail2Ban and make their ban permanent. This allows us to limit the constant emails one might receive about the same people trying to compromise our system(s). For those who aren’t using the emailing portion of Fail2Ban; this script still greatly takes the load off of Fail2Ban because it no longer has to manage the constant repeat offenders. Our logs are less cluttered too.

The script will address several things:

  1. It will handle multiple attacks from people within the same Class C type netmask.
  2. It will allow for the permanent bans to stick even after your system is rebooted. Unlike Fail2Ban’s list of blocked perpetrators, which is lost if the system (or iptables) is restarted.
  3. It will enforce the use of iptable’s DROP directive instead of REJECT. This is a slightly more secure approach in handling those who aren’t ever allowed to come back.
  4. It will support the fact that over time, you may want to add and remove from this new ban list I keep speaking of. Basically you can re-run the steps outlined in this blog again (and again) and not lose the addresses you’ve already blocked.
  5. The script maintains a global list of addresses in a simple delimited format. You can then choose to share this list with other systems (or colleagues) to block the same unwanted users on these systems too.

Script Dependencies and Requirements

For this script to work, you can virtually use any Linux/FreeBSD/Unix operating system as long as you’re also using fail2ban in conjunction with iptables.

The script makes use of sed, and gawk to massage the data. These tools are common an available to all operating systems. But not all of them necessarily have them installed by default. Make sure you’ve got them before proceeding:

# Fedora / CentOS 4,5,6 / RedHat 4,5,6
yum -i install gawk sed

# Ubuntu / Debian
apt-get install gawk sed

The Goods

Without further ado, the below code is documented quite heavily so that you can just copy the sections into your terminal screen as the systems root user. Don’t try the next section until you’re done with the previous.

Although this code works for me, I still must caution you all the same: I will not be held liable for any loss, damage or expense as a result of the actions taken by my script. I’ve only used it without problem with CentOS 6.x. That said, the simplicity of it should make it work with any other *nux based OS as well.

# Author: Chris Caron
# Date: Tue, Apr 7th, 2015
########################################
# Environment Variables
########################################
# YOU MUST CORRECTLY SET THESE OR THE REMAINING FUNCTIONS OF
# THIS CODE WILL NOT WORK CORRECTLY.
#
# THIS SCRIPT ASSUMES YOU ARE RUNNING AS 'root'
#
# Public Ethernet Device
# For my home system, i have this set to ppp0; You'll want to
# set this to the Ethernet device that harm can venture from.
# such as the internet.
PUBDEV=eth0

# The name of the iptables chain to manage the permanent bans
# within. The name doesn't matter so long that it doesn't
# collide with a chain you're already managing.
# Also, Do not change this value in the future because that
# will hinder the ability to upgrade/append new bans easily.
# It is doubtful that it's current set value will conflict
# with anything. Therefore just leave the name the way it is
# now:
FILTER=nuxref-perm-ban

# The script makes an effort to detect IP Addresses all
# coming from the same Class C type network.  Rather then
# have an entry per IP like fail2ban manages; we group
# them into 1 to speed the look-ups iptables preforms.
# You'll want to identify the minimum number of IP
# addresses matched within the same alike (Class C) network
# before this grouping takes place.
CLASSCGRP_MIN=2

# IP Tables configuration file read during your system
# startup.
IPTABLES_CFG=/etc/sysconfig/iptables

# IPTables Chain Index
# Identify where you want your ban list to be applied
# To reduce overhead, the banlist should be processed 'after'
# some core checks you do.  For example I have a series of other
# checks i do first such as allowing already established
# connections to pass through.  I didn't want the ban list
# being applied to these, so for my system, i set this to 11.
# Setting it to 1 is safe because it guarantees it's at least
# processed first on systems who don't actively maintain their
# own custom firewall list.
CHAINID=1

########################################
# Preparation
########################################

# First we built a massive sorted and unique list of
# already banned IP addresses.
# The below is clever enough to include previous content
# you've banned from before (if any exists):
(
   # carry over old master list
   iptables -L -n | awk "/Chain $FILTER/, $NF ~ /RETURN/" | 
    egrep DROP | sed -e 's/^[^0-9.]*([0-9]+.[0-9]+.[0-9]+).([0-9/]+).*/1 .2/g'
   # update master list with new data
   iptables -L -n | egrep REJECT | 
    sed -e 's/^[^0-9.]*([0-9]+.[0-9]+.[0-9]+).([0-9]+).*/1 .2/g'
) | sort -n | uniq > list
 
# Now we build a separate list (in memory) that will track
# all of the ip addresses that met our $CLASSCGRP_MIN flag.
CLASSC_LIST=$(cat list | cut -f1 -d' ' | uniq -c | 
    sed 's/^ *//g' | sort -n | 
    awk "int($1)>=$CLASSCGRP_MIN" | cut -f2 -d' ')
 
# We eliminate the duplicates already pulled into memory
# from the master list of IPs
for NW in $CLASSC_LIST; do sed -i -e "/^$NW /d" list; done

# Now we scan our list and repopulate them into our master
# list. We place these entries at the head of the file so
# that they'll be added to our iptable ban chain first.
for NW in $CLASSC_LIST; do sed -i "1s/^/$NW .0/24n/" list; done

# Using our list of banned IP addresses, we now generate
# the actual iptable entries (into a file called
# 'commands':
(
   # Creates the chain
   echo iptables -N $FILTER
 
   # Build List of Addresses using our list file
   while read line; do           
      IP=$(echo $line | tr -d '[:space:]')
      echo iptables -A $FILTER -s $IP -j DROP;
   done < list
 
   # Allow future iptable processing to continue 
   # after reading through this chain by appending
   # the RETURN action.
   echo iptables -A $FILTER -j RETURN
 
   # Add chain to INPUT
   echo iptables -t filter -I INPUT $CHAINID -i $PUBDEV -j $FILTER
) > commands

########################################
# IPTables (Temporary Instalment)
########################################

# Have a look at our commands if you want:
cat commands

# Apply these new rules now with the following command:
sh commands

# The commands generated in the 'commands' text file 
# are only temporary; they will be lost if your
# machine (or iptables) is ever restarted

########################################
# IPTables (Permanent Installation)
########################################
# Consider making a backup of your configuration in case you
# need to roll back
/bin/cp -f $IPTABLES_CFG $IPTABLES_CFG.backup

# Now we generate all of the commands needed to place
# into our iptables configuration file:
sed -e 's/^iptables[ ]*//g' commands | 
    egrep "^-A $FILTER -s " > commands-iptables
# Clean up old configuration
sed -i -e "/^:$FILTER -/d" "$IPTABLES_CFG"
sed -i -e "/^-A INPUT .* $FILTER$/d" "$IPTABLES_CFG"
sed -i -e "/^-A $FILTER -/d" "$IPTABLES_CFG"
 
# Now push all of our new ban entries into the iptables file
sed -i -e "/:OUTPUT .*/a:$FILTER - [0:0]" "$IPTABLES_CFG"
sed -i -e "/:$FILTER - .*/a-A INPUT -i $PUBDEV -j $FILTER" "$IPTABLES_CFG"
sed -i -e "/-A INPUT -i $PUBDEV -j $FILTER.*/r commands-iptables" "$IPTABLES_CFG"

# Now preform the following to reset all of the fail2ban
# jails you've got as well load your new permanent ban setup
service fail2ban stop
service iptables restart
service fail2ban start

# If you have a problem; just roll back your backup you
# created and rerun the 3 commands above again. You can
# have a look at the table with the following command:
iptables -L -n -v

########################################
# IPTables (Optional Tiding)
########################################
# the above will insert the banlist at the top
# The below will just correct this and move it
# clean entry(s) from INPUT
(
   while [ 1 -eq 1 ]; do
      ID=$(iptables -nL --line-numbers | 
           egrep "^[0-9]+[ t]+$FILTER " | 
           head -n 1 | 
           sed -e 's/^([0-9]+).*/1/g')
      [ -z "$ID" ] && break;
      iptables -D INPUT $ID
   done
   # Re insert at the correct $CHAINID
   iptables -t filter -I INPUT $CHAINID -i $PUBDEV -j $FILTER
)

# have another look if you want (if you tidied)
iptables -L -n -v

########################################
# Cleanup (Optional)
########################################
# Remove temporary files when your done; or save them if you
# want to port this data to another server:
rm -f list commands commands-iptables

You can undo and destroy the new entries an any time using the following:

# disassociate filter from INPUT
while [ 1 -eq 1 ]; do
   ID=$(iptables -nL --line-numbers | 
        egrep "^[0-9]+[ t]+$FILTER " | 
        head -n 1 | 
        sed -e 's/^([0-9]+).*/1/g')
   [ -z "$ID" ] && break;
   iptables -D INPUT $ID
done
# flush filter
iptables -F $FILTER
# remove filter
iptables -X $FILTER

Credit

This blog took some time to put together and test! If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Sources

There were not many sources used to make this entry. Most of it is just shell scripting knowledge I’ve adopted over the years mixed with some iptable commands. Here are some sources anyway that are applicable: