Tag Archives: python

Apprise Makes Sending Notifications via the CLI Easy

Introduction

Apprise is a free Python based notification tool that just makes our lives easier. It allows you to interact with just about all of the most popular Notification Services available to us today! I mean right now while writing this blog, there are already 28 supported services you can use! We’re talk about services like Discord, Telegram, Slack, Pushbullet, IFTTT, Amazon Web Service (SNS), Email, etc, etc.

How Does It Work?

You could be sending our system stats via an email from the command line:

# top will print all of the processes running on our Linux system
# Then we pipe that result into apprise and send it in an email:
top -b -n 1 -c | \
   apprise -t "my system stats" mailto://user:password@hotmail.com

Or maybe you’re a Microsoft Windows user! No problem, Apprise is not OS dependent at all! You could still send your services running to your email like so:

# TASKLIST will print all of the services running on our Widows system
# Then we pipe that result into apprise and send it in an email:
TASKLIST /svc ^ \
   apprise -t "my system stats" mailto://user:password@hotmail.com

The tool itself is pretty simple to use:

apprise [Options] [[[URI1] [URI2] ...]

Here are what the options look like:

Option Description
-t, –title TEXT Specify the message title. This field is optional.
-b, –body TEXT Specify the message body. If you don’t specify this value, then STDIN is used instead.
-n, –notification-type TYPE Specify the message type (default=info). The other options are success, warning, and error.
-T, –theme THEME Specify the default theme. This isn’t discussed in this blog entry, but I’ll talk about it soon!
-v, –verbose By default apprise is pretty quiet. By specifying -v (or a few more -vv and -vvv) you can increase the verbosity of the script. This is useful if you’re trying to figure out why a service might not be working for you.

To make a notification you must provide:

  1. One or more Uniform Resource Identifier(s) (URI). These are used to identify and configure Apprise with the service you wish to access (Email, Discord, etc). I’ll explain a more about this in the next section.
  2. A message body to pass along with the notification you’re going to make. This could be piped in from STDIN like in the examples above, or you can pass the message in via the –body (-b) switch.

Note: In the not so distant future, the message body requirement won’t be required for all services as it is today. An example where the body isn’t always necessary is for IFTTT requests. The message body also isn’t required for most home automation services that Apprise may support in the future!

The Apprise Notification URI

The Uniform Resource Identifier (URI) is the magic one-liner configuration that tells Apprise everything it needs to in order to notify your service(s).

Generally URI looks like this:

  • service://user@host:port
  • service://password@host:port
  • service://user:password@host:port
  • service://host:port
  • service://host

We see these every day when we access a website such as http://nuxref.com.

Take an Apprise Email URI for example:

mailtos://nuxref:mypass@gmail.com
  ^         ^      ^       ^
  |         |      |       |
service   login    |   email domain
                   |
               password

In the above example, Apprise knows you want to send an email because of the mailto:// service identified in front. From there it can acquire all of the server information it needs about this service based upon the rest of the URI provided. Email (URI) configuration can get complicated depending on what you want to do… But for most email providers, Apprise makes it really easy! You can read more about email URIs here.

There are other Apprise URIs that are really easy to use too. For example, if you’re on a Microsoft Windows PC, you can use the windows:// URI and send your system a broadcast message without any effort at all:

apprise -t "Apprise Is Awesome" -b "A Windows Notification" windows://

Note: This only works on a Microsoft Windows PC. It also requires you to have the pypiwin32 bundle installed (if you don’t have it already).

If you’re on a Linux PC you can use the gnome:// or kde:// to achieve the same effects as The MS Windows users get:

# Gnome Messages:
apprise -t "Apprise Is Awesome" -b "A Gnome Notification" gnome://

# KDE Messages
apprise -t "Apprise Is Awesome" -b "A KDE Notification" kde://

There are so many supported notification services, and the list continues to grow.

The other advantage of having all of your configuration in a single URI is that can chain them all together. Apprise will notify each and every one you specify with the message you put in place. There is no limit to the number of servers that you specify.

# The below fires off a pushbullet (pbul://) notification
# and an email (mailto://)
apprise -b "visit nuxref.com" \
   mailto://myusername:mypassword@yahoo.com \
   pbul://o.gn5kj6nfhv736I7jC3cj3QLRiyhgl98b

By uniquely mapping services to their own URIs, we can additionally allow for custom options this way too. For example, let’s say we want our email to be text formatted instead of the default html. No problem:

# Send all of our log files in a text based email format:
cat /var/log/messages | apprise -t "Logging" \
   mailtos://nuxref:mypass@gmail.com?format=text 

Notification Types

Notification Type Image
info
success
warning
failure

By default all of your notifications are sent as an info message. This comes into play depending on what kind of notification service you’re using. Some services really don’t care, while others will give you a richer experience as a result of knowing the message type.

Notification services like Slack and Telegram can decorate the message you send it with an icon.

Meanwhile other services like SMS Messages (sent from AWS) or Emails won’t use this feature at all.

# The below sends a success message to the general channel on
# slack (slack://):
apprise -b "visit nuxref.com" -t "success" \
   slack:///T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F/#general

Note: You will need to build your own URI for your own services that you use. the URIs I use in my example are made up just to demonstrate how it works.

Wait… Exposing My Password On The Command Line Is a Terrible Idea!

I couldn’t agree more! Thankfully Apprise supports configuration files

# Place your URLs into a configuration file in our home directory:
cat << _EOF > ~/.apprise
mailto://myusername:mypassword@yahoo.com
pbul://o.gn5kj6nfhv736I7jC3cj3QLRiyhgl98b
slack:///T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F/#nuxref
_EOF

# Now secure the file so no one can peak at what is personal
# to you:
chmod 400 ~/.apprise

# Now we can call apprise and source the file for our service URIs:
# By default apprise looks in ~/.apprise so now the command line
# gets even easier to use (and more secure too!)
apprise -b 'Apprise Rocks!'

# The above command would fire the notification to all 3 of the
# services you identified in the ~/.apprise file.

# If you want to manage multiple files, you can easily do this too
# just use the --config switch on the command line and point to
# your file:
apprise -b 'My Message Body' --config=/path/to/config

Microsoft Windows users could do a similar solution:


# Prepare ourselves a file (use a text editor is more ideal
# then what I'm putting here):
mkdir  %APPDATA%/Apprise
echo "mailto://myusername:mypassword@yahoo.com" > %APPDATA%/Apprise/apprise
echo "pbul://o.gn5kj6nfhv736I7jC3cj3QLRiyhgl98b" >> %APPDATA%/Apprise/apprise
echo "slack:///T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F/#nuxref" >> %APPDATA%/Apprise/apprise

# Set permissions on this file at this point if you want

# Now we can call apprise and source the file for our service URIs:
apprise -b 'Apprise Rocks!'

# If you want to manage multiple files, you can easily do this too
# just use the --config switch on the command line and point to
# your file:
apprise -b 'My Message Body' --config=C:\path\to\config

How Do I Get Apprise?

Assuming you’ve got Python (2.7 or 3.x) already installed, then PyPI is probably the best place to go. But if you have pip installed, just could also just do the following:

pip install apprise

That’s it! You’re ready to go!

Sources

SABnzbd Installation for CentOS 7.x

Introduction

SABnzbd is a versatile tool written in Python specifically designed to take an NZB-File as input and then retrieve all of the content defined inside of it for you automatically. Recently there have been new improvements with SABnzbd in the past few releases. So I thought I’d share a working out of the box solution using RPMs. In addition to this, I’ve packaged up my notification script I wrote which can further enhance this great product.

SABnzbd and Usenet

First off; lets summarize Usenet for those who don’t use it or aren’t already familiar with it. In the simplest terms: Usenet is a great big fucking mess; a total disaster. It’s basically a location where literally anyone with access to it can post/upload all they want at their free will anonymously. Don’t get me wrong; this is cool too because it’s basically one very large hard drive without anyone telling you what you can and can’t place onto it. Seriously though, just consider for a second what your computer would look like if you allowed absolutely anyone who has access to the internet to upload content freely and without rules to it. You’d probably run out of hard disk space quickly; and you’ll be left with a lot of content everywhere. Well; that’s exactly what Usenet is (greatly simplified); it’s one large centralized location filled with petabytes of data. The thing with Usenet though is… it isn’t, and won’t be running out of disk space anytime soon.

The good news is, (thankfully) people sift through the heaps of information constantly being posted (onto Usenet) using automated tools regularly. These tools that do the sifting are generally known as Indexers. I talk more about this in another blog I wrote here a while back. But basically whenever an Indexer finds some useful data, it records the location as to where it was found. Indexers record this information in a special file call an NZB-File. NZB-Files are effectively treasure maps containing the coordinates on Usenet to which a specific piece of data can be located at.

NZB-Files are like Treasure Maps

NZB-Files are like Treasure Maps

It should be known that if you’re planning on posting/uploading stuff to Usenet yourself (for backup purposes or what have you), most (good) software will generate you an NZB-File afterwards allowing you to retrieve your data back again later on.

Now lets throw SABnzbd into the picture because it specifically is designed to retrieve content based on an NZB-File it’s provided. The process looks a little like this:

SABnzbd Download Process

SABnzbd Download Process

  1. Acquire NZB-File: Presumably you already have it because you’re just trying to retrieve data you posted in the past. But alternatively, the NZB-File could have also been acquired from and Indexer too.
  2. Give SABnzbd Access to NZB-File: You simply hand off your treasure map to the application who’s actually going to go out there and get the content for you.

    This is truly the bread and butter of SABnzbd. The next set of steps identified below are all fully automated (and done behind the scenes) for you. They require nothing from you, but it’s worth explaining it for those who are interested.

  3. SABnzbd Establishes a Connection to Usenet: The catch with SABnzbd is it’s merely a vessel for fetching the data. You need to have provided it access to Usenet to which it will fetch this data from. Thus SABnzbd must (obviously) connect to a Usenet provider in order to retrieve data from Usenet itself. This step is put in place to make sure you’ve got yourself an account with a provider!
  4. SABnzbd Downloads Content: Segment by segment, each portion of your data is downloaded and re-assembled. There are lots of additional things that go on too such as making sure the data isn’t damaged and attempting to correct it (all automatic) if it is.
  5. SABnzbd Saves Data:If all goes well; and in most circumstances it will. You will have all of the content successfully retrieved at this point!

SABnzbd has tons of automation built into it; I’m really just focusing on the basics here to get you going.

I Want To Try It

Of course you do! So here’s the thing; if you’ve already connected to my repository here, you can get it by simply typing the following:

# Download and install SABnzbd plus the Notification Addon
# on CentOS 7.x using the nuxref repositories located
# here: http://nuxref.com/repo/
yum install -y --enablerepo=nuxref \
               --enablerepo=nuxref-shared \
               sabnzbd sabnzbd-script-notify

You can also just visit the location I host these packages directly (via my repository) here and download the RPMs for yourself:

Note: I provided the source rpms optionally; they are not required unless you want to build this for yourself from scratch.

SABnzbd Environment

It’s worth giving you a quick rundown of how the RPM installs itself upon your computer: First off, it creates a general user/group called sabnzbd. By default this is the user/group it will run as. To grant a user access to all of the content retrieved by SABnzbd, you can just add yourself to the sabnzbd group:

# as root; we can add ourselves to the sabnzbd group
usermod -a -G sabnzbd myuserid

# Note: if this user is logged in, they will have to log out and log
#       back in to have these new security credentials noticed. 

You can start it up with the command:

# as root; start up SABnzbd:
systemctl start sabnzbd.service

You’ll be able to access the web page through your browser by punching in http://localhost:9080.

You can set it up to survive reboots with the following command:

# as root; start up SABnzbd when the machine is first
# powered on:
systemctl enable sabnzbd.service
  • All of your log files will show up in /var/log/sabnzbd/sabnzbd.log
  • All of configuration will get written to /etc/sabnzbd/sabnzbd.conf
  • All of the variable data (file processing, etc) will be located in /var/lib/sabnzbd/*. In fact this is a very important directory because unless you configure things differently, all downloaded content will appear in /var/lib/sabnzbd/complete.

Firewall Configuration

The package will provide you the files needed to set up the firewall and make SABnzbd available to you from other stations by simply doing the following:

# as root; start up SABnzbd when the machine is first
# powered on (assuming your network is set up to the `home` zone
firewall-cmd --zone=home --add-service sabnzbd --permanent

# Now reload your firewall to take on the new change:
firewall-cmd --reload

SABnzbd Notifications

SABnzbd can keep you posted on what it’s doing by sending you emails when a download completes (or fails). It can send you an notification on Pushbullet, and a few others too.

If you want to use the Notify script I wrote, you’re already almost set up and ready to go because it’s in the sabnzbd-script-notify rpm you already installed.

To enable it, you simply need to need to access the Notifications tab from within the SABnzbd Configuration section.

SABnzbd Notification Setup

SABnzbd Notification Setup

  1. Select the Enable notification script checkbox
  2. Select the Notify.py script from the dropdown list next to the Script category
  3. Next to the Parameter category, you must specify the URL(s) identifying which service(s) you want to notify.

    Depending on what you want plan on alerting, the URL(s) you specify in the Parameter field will vary. You can get a better understanding of the URL options supported here.

Credit

This blog took me a very long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Special thanks to Safihre for reaching out to me and allowing me to contribute to their product with the notification add-on.

Sources

Linux Administration Tools & Tricks

Introduction

All *nux administrators have tools they use to make their job easier. This blog will be an ongoing one where every time I find or create a handy tool, I’ll share it. I’ll do my best to provide a quick write up with it and some simple examples as to how it may be useful to you too.
Currently, this blog touches on the following:

Applications

Please note that most of these tools can be directly retrieved from my repository I’m hosting. You may wish to connect to it to make your life easier.


genssl: SSL/TLS Certificate Generator and Certificate Validation

genssl is a simple bash script I created to automatically generate Private/Public key pairs that can be used by all sorts of applications requiring SSL/TLS encryption. This could be used by any number of servers you’re hosting (Web, Mail, FTP, etc). It will also test previously created keys (validating them). It supports generating keys that are to be signed by Certificate Authority as well as self signed and test keys too.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref genssl

What You’ll Need To Know
It’s really quite simple, SSL/TLS keys are usually assigned to domain names. Thus this is the only input to the tool in addition to the key type you want to generate (self-signed, test, or registered):

# Generate a simple self-signed Certificate Private/Public
# key pair for the domain example.com you would just type the
# following:
genssl -s example.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README

# You can specify as many domain names as you want to
# generate more then one:
genssl -s example.com nuxref.com myothersite.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README
#   - nuxref.com.key
#   - nuxref.com.crt
#   - nuxref.com.README
#   - myothersite.com.key
#   - myothersite.com.crt
#   - myothersite.com.README

You can verify the keys with the following command:

# The following scans the current directory
# for all .key, .crt, and/or .csr files and tests that
# they all correctly match against one another.
genssl -v

If you need a signed certificate by a registered Certificate Authority, genssl will get you half way there. It will generate you your public key and an unsigned certificate (.csr). You’ll then need to choose a Certificate Authority that works best for you (cost wise, features you need). You will provide them your unsigned certificate you just generated and in return, they’ll produce (and provide) for you the private key you need (.key). Signed (registered) certificates cost money because you’re paying for someone else to confirm the authenticity of your site when people visit it. This is by far the absolute best way to publicly host a secure website!

# Generate an unsigned certificate and public key
# pair for the domain example.com with the following:
genssl -r example.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README

You’ll notice that you also generate a README file for every domain you create keys for. Some of us won’t generate keys often; so this README provides some additional information that will explain how you can use the key. It lists a few Certificate Authoritative locations you can contact, and provides examples on how you can install the key into your system. Depending on the key pair type you generate, the README will provide you with information specific to your cause. Have a look at the contents yourself after you generate a key! I find the information in it useful!

The last thing worth noting about this tool is to create a key pair, you usually provide information about yourself or the site you’re creating the key for. I’ve defaulted all these values to ones you’re probably not interested in using. You can over-ride them by specifying additional switches on the command line. But the absolute easiest (and better) way of doing it, is to just provide a global configuration file it can reference which is /etc/genssl. This file is always sourced (if present). The next file that is checked and sourced is ~/.config/genssl and then ~/.genssl.
Set this with information pertaining to your hosting to ease the generation of of the key details. Here is an example of what any one of these configuration files might look like:

# The country code is represented in it's 2 letter abbreviated version:
# hence: CA, US, UK, etc
# Defaults to "7K" if none is specified
GENSSL_COUNTRY="7K"
# Your organization/company Name
# Defaults to "Lannisters" if none is specified
GENSSL_ORG="Lannisters"
# The Province and or State you reside in
# Defaults to "Westerlands" if none is specified
GENSSL_PROVSTATE="Westerlands"
# Identify the City/Town you reside in
# Defaults to "Casterly Rock" if none is specified
GENSSL_LOCATION="Casterly Rock"
# Define a department; this is loosely used. Some
# just leave it blank
# Defaults to "" (blank) if not is specified
GENSSL_DEPT=""

There are other variables you can override, but the above are the key ones. The default cipher strength is rsa:2048 and keys expire after 730 days (which equates to 2 years). This is usually the max time Certificate Authoritative sites will sign your key for anyway (if registering with them).

You don’t need a configuration file at all for this script to work, there are switches for all of these variables too that you can pass in. Its important to note that passed in switches will always over-ride the configuration file too, but this is the same with most applications anyway.

Some Further Reading


fencemon: A System Monitoring Tool

fencemon is a beautiful creation by Red Hat, the only thing I did was package it as an RPM. I don’t think it’s advertised or used enough by system administrators. It simply runs as a cron and constantly gathers detailed system information. Information such as the current programs running, the system memory remaining, network statistics, disk i/o, etc. It captures content every 10 seconds but does so by using tools that are not i/o intensive. So running this will not slow your machine down enough to justify not using it.

Where this information becomes vital is during a system failure (in the event one ever does occur). It will allow you to see exactly what the last state of the system was (processes in memory, etc) prior to your issue. The detailed information can be used with everyday troubleshooting as well too such as identifying that process that is going crazy overnight and other anonymities that you’re never around to witness first hand.

Why is it called fencemon?
It got it’s name because it’s initial purpose was to run in clustered environments which do a think called ‘fencing’. It’s when another cluster node sees that another is violating some of the simple cluster rules put in place, or it simply isn’t responding anymore. Fencing is effectively the power cycling of the node (so an admin doesn’t have to). In almost all cases (unless there is seriously something wrong with the fenced node), it will reboot and rejoin the cluster correctly. This tool would have allowed an admin to see the final state of the cluster node before it was lost. But any server can crash or go crazy when deploying software in a production environment. We all hope it never happens, but it can. The logging put in place by fencemon can be a life saver!

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref fencemon

What You’ll Need To Know
The constant system capturing that is going on will report itself to /var/log/fencemon/.

The format of the log files is:
hostnameYYYYMMDDHHMMSS-info-capturetype.log

Once an hour elapses, a new file is written to. Each file contains system statistics in 20 second bursts; as you can imagine, there is A LOT of information here.

the following capturetype log files (with their associated commands) are gathered in 20 second increments:

  • hostnameYYYYMMDDHHMMSS-info-ps-wchan.log:
    ps -eo user,pid,%cpu,%mem,vsz,rss,tty,stat,start,time,wchan:32,args
  • hostnameYYYYMMDDHHMMSS-info-vmstat.log:
    vmstat
  • hostnameYYYYMMDDHHMMSS-info-vmstat -d.log:
    vmstat-d
  • hostnameYYYYMMDDHHMMSS-info-netstat.log:
    netstat -s
  • hostnameYYYYMMDDHHMMSS-info-meminfo.log:
    cat /proc/meminfo
  • hostnameYYYYMMDDHHMMSS-info-slabinfo.log:
    cat /proc/slabinfo
  • hostnameYYYYMMDDHHMMSS-info-ethtool-$iface.log:
    # $iface will be whatever is detected on your machine
    # you can also define this variable in /etc/sysconfig/fencemon
    # ifaces="lo eth0"
    # The above definition would cause the following to occur:
    /sbin/ethtool -S lo >> 
       /var/log/fencemon/hostname-YYYYMMDD-HHMMSS-info-ethtool-lo.log
    /sbin/ethtool -S eth0 >> 
       /var/log/fencemon/hostname-YYYYMMDD-HHMMSS-info-ethtool-eth0.log
    

The log files are kept for about 2 days which can occupy about 750MB of disk space. But hey, disk space is cheap now of days. You should have no problem reserving a GIG of disk space for this info. It’s really worth it in the long run!

Some Further Reading


datemath: A Date/Time Manipulation Utility

I won’t go too deep on this subject since I’ve already blogged about it before here. In a nutshell, It can easily provide you a way of manipulating time on the fly to ease scripting. This tool is not rocket science by any means, but it simplifies shell scripting a great deal when preforming functionality that is time sensitive.

This is effectively an extension or the the missing features to the already existing date tool which a lot of developers and admins use regularly.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref datemath

What You’ll Need To Know
There are 2 key applications; datemath and dateblock. Datemath can ease scripting by returning you time relative to when it was called; for example:

# Note: the current date/time is printed to the
#        screen to give you an idea how the math was
#        applied.
# date +'%Y-%m-%d %H:%M:%S' prints: 2014-11-14 21:49:42
# What will the time be in 20 hours from now?
datemath -o 20
# 2014-11-15 17:49:42

# Top of the hour 9000 minutes in the future:
datemath -n 9000 -f '%Y-%m-%d %H:00:00'
# 2014-11-21 03:00:00

If you use a negative sign, then it looks in the past. There are lots of reasons why you might want to do this; consider this little bit of code:

# Touch a file that is 30 seconds older than 'now'
touch -t $(datemath -s -30 -f '%Y%m%d%H%M.%S') reference_file

# Now we have a reference point when using the 'find'
# command. The below command will list all files that
# at least 30 seconds old. This might be useful if your
# hosting an FTP server and don't want to process files
# until they've arrived completely.
find -type f -newer reference_file
# You could move the fully delivered files to a new
# directory with a slight adjustment to the above line
find -type f -newer reference_file -exec mv -f {} /new/path ;

# Consider this for an archive clean-up for a system
# you maintain. Simply touch a file as being 31 days
# older then 'now'
touch -t $(datemath -d -31 -f '%Y%m%d%H%M.%S') reference_file

# now you can list all the files that are older then
# 31 days.  Add the -delete switch and you can clean
# up old content from a directory.
find /an/archive/directory -type f ! -newer reference_file

# You can easily script sql statements using this tool too
# consider that you need to just select yesterdays data
# for an archive:
START=$(datemath -d -1 -f '%Y-%m-%d 00:00:00')
FINISH=$(date +'%Y-%m-%d 00:00:00')
BACKUP_FILE=/absolute/path/to/put/backup/$START-backup.csv
# Now your SQL statement could be:
/usr/bin/psql -d postgres 
      -c "COPY (SELECT * FROM mysystem.data WHERE 
                mysystem.start_date >= '$START' AND 
                mysystem.finish_date < '$FINISH') 
          TO '$BACKUP_FILE';"
# Now that we've backed our data up, we can remove
# it from our database:
/usr/bin/psql -d postgres 
      -c "DELETE FROM mysystem.data  WHERE 
                mysystem.start_date >= '$START' AND 
                mysystem.finish_date < '$FINISH'"

Some Further Reading


dateblock: A Cron-Like Utility

dateblock allows us to mimic the functionality of sleep by blocking the process. The catch is dateblock blocks until a specific time instead of for a duration of time.

This very subtle difference can prove to be a very effective and powerful too, especially when you want to execute something at a specific time. Cron’s are fine for most scenarios, but they aren’t ideal in a clustered environment where this tool excels. In a clustered environment you may only want an action to occur on the node hosting the application, where a cron would require the action to fire on all nodes.

The python C++ library enables this tools usage even further since you can integrate it with your application.

Just like datemath is, dateblock is discussed in much more detail in my blog about it here.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref dateblock

# Get the python library too if you want it
yum install -y --enablerepo=nuxref python-dateblock

What You’ll Need To Know
Consider the following:

# block for 30 seconds...
sleep 30

# however dateblock -s 30 would block until the next 30th second
# of the minute is reached. Consider the time:
# date +'%Y-%m-%d %H:%M:%S' prints: 2014-11-14 22:14:16
# dateblock would block until 22:14:30 (just 14 seconds later)
dateblock -s 30

Dateblock works just like a crontab does and supports the crontab format too. It was written by me to specifically allow accuracy for time sensitive applications running. If you write your actions and commands in a script under a dateblock command, you can always guarantee your actions will be called at a precise time.

Since it supports cron entries you can feed it things like this:

# The following will unblock if the minute becomes
# divisible by 10.  so this would unblock at the
# :00, :10, :20, :30: 40: and :50 intervals
dateblock -n /10

# the above command could also be written like this:
dateblock -n 0,10,20,30,40,50

# You can mix and match switches to tweak the tool to
# always work in your favour.

A really nice switch that can help with your debugging and development is the –test (-t) switch. This will cause dateblock to ‘not’ block at all. Instead it will just spit out the current time, followed by the time it ‘would’ have unblocked at had you not put it into test mode. This is good for trying to tweak complicated arguments to get the tool to work in your favour.

# using the test switch, we can make sure we're going
# to unblock at time intervals we expect it to. In this
# example, i use the test switch and a cron formatted
# argument.  In this example, I'm asking it to unblock
# ever 2 days at the start of each day (midnight with
# respect to the systems time)
dateblock -t -c "0 0 0 /2"
Current Time : 2014-11-14 22:24:06 (Fri)
Block Until  : 2014-11-16 00:00:00 (Sun)

# Note: there is an extra 0 above that may through you
#       off.. in the normal cron world, the first
#       argument is the 'minute' entry.  Well since
#       dateblock offers high resolution, the first
#       entry is actually 'seconds', then minute, etc.

There are man pages for both of these tools. You’ll get a lot more detail of the power they offer. Alternatively, if you even remotely interested, check out e my blog entry on them.

The other cool thing is dateblock also has a python library I created.

# Import the module
from dateblock import dateblock

# Here is a similar example as above and we set
# block to false so we know how long were blocking until
unblock_dt = dateblock('0 0 0 /2', block=False)
print("Blocking until %s" % unblock_dt
              .strftime('%Y-%m-%d %H:%M:%S))
# by default, blocking is enabled, so now we can just
# block for the specified time
datetime.datetime('0 0 0 /2')

Some Further Reading


nmap (Network Mapper)

nmap is port scanner & network mapping tool. You can use it to find out all of the open ports a system has on it and you can also use it to see who is on your network. This tool is awesome, but it’s intentions can be easily abused. It’s common sense to scan your own network to make sure that only the essential ports are open to the outside world, but it’s not kind to scan a system to which you are not the owner of. Hence, this tool should be used to identify your systems weaknesses so that you can fix them; it should not be used to identify an others weakness. For this reason, if you do install nmap on your system, consider limiting permission so that only the root user has access to it.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref-shared nmap

# Restrict its access (optional, but ideal)
chmod o-rwx /usr/bin/nmap

What You’ll Need To Know
Now you can do things like:

  • Scan your system to identify all open ports:
    # Assuming the server you are scanning is 192.168.1.10
    nmap 192.168.1.10
    
  • Scan an entire network to reveal everyone on it (including MAC Address information):
    # Assuming a class C network (24 masked bits) with
    # a network address of 192.168.1.0
    nmap -n -v -sP 192.168.1.0/24
    

Note: Don’t panic if your system appears to have more ports open then you had thought. Well at least don’t panic until you determine where you are running your network map test from. You see, if you run nmap on the same system you are testing against, then there your test might not prove accurate since firewalls do not normally deny any local traffic. The best test is to use a second access point to test against the first. Also, if your server sits on more then one network, make sure to test it from a different server on each network it connects to!

Some Further Reading

  • A blog with a ton of different examples identifying things you can do with this tool.

lsof (LiSt Open Files)

This tool is absolutely fantastic when it comes to preforming diagnostics or handling emergency scenarios. This command is only available to the root user; this is intentional due to the power it offers the system administrators.

I’m not hosting this tool simply because it ships with most Linux flavors (Fedora, CentOS, Red Hat, etc). Therefore, the following should haul it into your system if it’s not already installed:

# just type the following:
yum install -y lsof

# If this doesn't work, the rpm will be found
# on your DVD/CD containing your Linux Operating
# System.

What You’ll Need To Know
This tool when executed on it’s own (without any options) simply lists every open file, device, socket you have on one great big list. This can be useful when paired with grep if your looking for something specific.

# list everything opened
lsof

But it’s true power comes from it’s ability to filter through this list and fetch specific things for you.

# list all of the processes utilizing port 80 on your system:
lsof -i :80 -t

# or be more specific and list all of the processes accessing
# a specific service you are hosting on an IP listening on
# port 80:
# Note: the below assumes we're interested in those accessing
#       the ip address 192.168.0.1
lsof -i @192.168.0.1:80 -t

# If you pair this with the kill command, you can easily kick
# everyone off your server this way:
kill -9 $(lsof -i @192.168.0.1:80 -t)

But the tool is also great for even local administration; let’s say there is a user account on your system with the login id of bob.

# find out what bob is up to and what he's accessing on
# your system this way (you may need to pair this with
# grep since the output can be quite detailed
lsof -u bob

# Perhaps you just want to see all of the network activity
# bob is involved in. The following spits out a really
# easy to read list of all of the network related processes
# bob is dealing with.
lsof -a -u bob -i

# You can kill all of the TCP/IP processes bob is using with
# the following (if bob was violating the system, this might
# be a good course of action to take):
kill -9 $(lsof -t -u bob)

Perhaps you have a file on your system you use regularly, you can find out who is also using it with the following command:

# List who is accessing a file
lsof /path/to/a/file

Some Further Reading

  • lsof Survival Guide: A Stack Overflow post with some awesome tricks you can do with this tool.
  • More great lsof examples. This is someones blog who specifically wrote about this tool specifically. They provide many more documented examples of this tools usage here.

Credit

This blog took me a very long time to put together and test! If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.