Category Archives: Linux Tools

Linux Administration Tools & Tricks

Introduction

All *nux administrators have tools they use to make their job easier. This blog will be an ongoing one where every time I find or create a handy tool, I’ll share it. I’ll do my best to provide a quick write up with it and some simple examples as to how it may be useful to you too.
Currently, this blog touches on the following:

Applications

Please note that most of these tools can be directly retrieved from my repository I’m hosting. You may wish to connect to it to make your life easier.


genssl: SSL/TLS Certificate Generator and Certificate Validation

genssl is a simple bash script I created to automatically generate Private/Public key pairs that can be used by all sorts of applications requiring SSL/TLS encryption. This could be used by any number of servers you’re hosting (Web, Mail, FTP, etc). It will also test previously created keys (validating them). It supports generating keys that are to be signed by Certificate Authority as well as self signed and test keys too.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref genssl

What You’ll Need To Know
It’s really quite simple, SSL/TLS keys are usually assigned to domain names. Thus this is the only input to the tool in addition to the key type you want to generate (self-signed, test, or registered):

# Generate a simple self-signed Certificate Private/Public
# key pair for the domain example.com you would just type the
# following:
genssl -s example.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README

# You can specify as many domain names as you want to
# generate more then one:
genssl -s example.com nuxref.com myothersite.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README
#   - nuxref.com.key
#   - nuxref.com.crt
#   - nuxref.com.README
#   - myothersite.com.key
#   - myothersite.com.crt
#   - myothersite.com.README

You can verify the keys with the following command:

# The following scans the current directory
# for all .key, .crt, and/or .csr files and tests that
# they all correctly match against one another.
genssl -v

If you need a signed certificate by a registered Certificate Authority, genssl will get you half way there. It will generate you your public key and an unsigned certificate (.csr). You’ll then need to choose a Certificate Authority that works best for you (cost wise, features you need). You will provide them your unsigned certificate you just generated and in return, they’ll produce (and provide) for you the private key you need (.key). Signed (registered) certificates cost money because you’re paying for someone else to confirm the authenticity of your site when people visit it. This is by far the absolute best way to publicly host a secure website!

# Generate an unsigned certificate and public key
# pair for the domain example.com with the following:
genssl -r example.com
# ^-- creates the following:
#   - example.com.key
#   - example.com.crt
#   - example.com.README

You’ll notice that you also generate a README file for every domain you create keys for. Some of us won’t generate keys often; so this README provides some additional information that will explain how you can use the key. It lists a few Certificate Authoritative locations you can contact, and provides examples on how you can install the key into your system. Depending on the key pair type you generate, the README will provide you with information specific to your cause. Have a look at the contents yourself after you generate a key! I find the information in it useful!

The last thing worth noting about this tool is to create a key pair, you usually provide information about yourself or the site you’re creating the key for. I’ve defaulted all these values to ones you’re probably not interested in using. You can over-ride them by specifying additional switches on the command line. But the absolute easiest (and better) way of doing it, is to just provide a global configuration file it can reference which is /etc/genssl. This file is always sourced (if present). The next file that is checked and sourced is ~/.config/genssl and then ~/.genssl.
Set this with information pertaining to your hosting to ease the generation of of the key details. Here is an example of what any one of these configuration files might look like:

# The country code is represented in it's 2 letter abbreviated version:
# hence: CA, US, UK, etc
# Defaults to "7K" if none is specified
GENSSL_COUNTRY="7K"
# Your organization/company Name
# Defaults to "Lannisters" if none is specified
GENSSL_ORG="Lannisters"
# The Province and or State you reside in
# Defaults to "Westerlands" if none is specified
GENSSL_PROVSTATE="Westerlands"
# Identify the City/Town you reside in
# Defaults to "Casterly Rock" if none is specified
GENSSL_LOCATION="Casterly Rock"
# Define a department; this is loosely used. Some
# just leave it blank
# Defaults to "" (blank) if not is specified
GENSSL_DEPT=""

There are other variables you can override, but the above are the key ones. The default cipher strength is rsa:2048 and keys expire after 730 days (which equates to 2 years). This is usually the max time Certificate Authoritative sites will sign your key for anyway (if registering with them).

You don’t need a configuration file at all for this script to work, there are switches for all of these variables too that you can pass in. Its important to note that passed in switches will always over-ride the configuration file too, but this is the same with most applications anyway.

Some Further Reading


fencemon: A System Monitoring Tool

fencemon is a beautiful creation by Red Hat, the only thing I did was package it as an RPM. I don’t think it’s advertised or used enough by system administrators. It simply runs as a cron and constantly gathers detailed system information. Information such as the current programs running, the system memory remaining, network statistics, disk i/o, etc. It captures content every 10 seconds but does so by using tools that are not i/o intensive. So running this will not slow your machine down enough to justify not using it.

Where this information becomes vital is during a system failure (in the event one ever does occur). It will allow you to see exactly what the last state of the system was (processes in memory, etc) prior to your issue. The detailed information can be used with everyday troubleshooting as well too such as identifying that process that is going crazy overnight and other anonymities that you’re never around to witness first hand.

Why is it called fencemon?
It got it’s name because it’s initial purpose was to run in clustered environments which do a think called ‘fencing’. It’s when another cluster node sees that another is violating some of the simple cluster rules put in place, or it simply isn’t responding anymore. Fencing is effectively the power cycling of the node (so an admin doesn’t have to). In almost all cases (unless there is seriously something wrong with the fenced node), it will reboot and rejoin the cluster correctly. This tool would have allowed an admin to see the final state of the cluster node before it was lost. But any server can crash or go crazy when deploying software in a production environment. We all hope it never happens, but it can. The logging put in place by fencemon can be a life saver!

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref fencemon

What You’ll Need To Know
The constant system capturing that is going on will report itself to /var/log/fencemon/.

The format of the log files is:
hostnameYYYYMMDDHHMMSS-info-capturetype.log

Once an hour elapses, a new file is written to. Each file contains system statistics in 20 second bursts; as you can imagine, there is A LOT of information here.

the following capturetype log files (with their associated commands) are gathered in 20 second increments:

  • hostnameYYYYMMDDHHMMSS-info-ps-wchan.log:
    ps -eo user,pid,%cpu,%mem,vsz,rss,tty,stat,start,time,wchan:32,args
  • hostnameYYYYMMDDHHMMSS-info-vmstat.log:
    vmstat
  • hostnameYYYYMMDDHHMMSS-info-vmstat -d.log:
    vmstat-d
  • hostnameYYYYMMDDHHMMSS-info-netstat.log:
    netstat -s
  • hostnameYYYYMMDDHHMMSS-info-meminfo.log:
    cat /proc/meminfo
  • hostnameYYYYMMDDHHMMSS-info-slabinfo.log:
    cat /proc/slabinfo
  • hostnameYYYYMMDDHHMMSS-info-ethtool-$iface.log:
    # $iface will be whatever is detected on your machine
    # you can also define this variable in /etc/sysconfig/fencemon
    # ifaces="lo eth0"
    # The above definition would cause the following to occur:
    /sbin/ethtool -S lo >> 
       /var/log/fencemon/hostname-YYYYMMDD-HHMMSS-info-ethtool-lo.log
    /sbin/ethtool -S eth0 >> 
       /var/log/fencemon/hostname-YYYYMMDD-HHMMSS-info-ethtool-eth0.log
    

The log files are kept for about 2 days which can occupy about 750MB of disk space. But hey, disk space is cheap now of days. You should have no problem reserving a GIG of disk space for this info. It’s really worth it in the long run!

Some Further Reading


datemath: A Date/Time Manipulation Utility

I won’t go too deep on this subject since I’ve already blogged about it before here. In a nutshell, It can easily provide you a way of manipulating time on the fly to ease scripting. This tool is not rocket science by any means, but it simplifies shell scripting a great deal when preforming functionality that is time sensitive.

This is effectively an extension or the the missing features to the already existing date tool which a lot of developers and admins use regularly.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref datemath

What You’ll Need To Know
There are 2 key applications; datemath and dateblock. Datemath can ease scripting by returning you time relative to when it was called; for example:

# Note: the current date/time is printed to the
#        screen to give you an idea how the math was
#        applied.
# date +'%Y-%m-%d %H:%M:%S' prints: 2014-11-14 21:49:42
# What will the time be in 20 hours from now?
datemath -o 20
# 2014-11-15 17:49:42

# Top of the hour 9000 minutes in the future:
datemath -n 9000 -f '%Y-%m-%d %H:00:00'
# 2014-11-21 03:00:00

If you use a negative sign, then it looks in the past. There are lots of reasons why you might want to do this; consider this little bit of code:

# Touch a file that is 30 seconds older than 'now'
touch -t $(datemath -s -30 -f '%Y%m%d%H%M.%S') reference_file

# Now we have a reference point when using the 'find'
# command. The below command will list all files that
# at least 30 seconds old. This might be useful if your
# hosting an FTP server and don't want to process files
# until they've arrived completely.
find -type f -newer reference_file
# You could move the fully delivered files to a new
# directory with a slight adjustment to the above line
find -type f -newer reference_file -exec mv -f {} /new/path ;

# Consider this for an archive clean-up for a system
# you maintain. Simply touch a file as being 31 days
# older then 'now'
touch -t $(datemath -d -31 -f '%Y%m%d%H%M.%S') reference_file

# now you can list all the files that are older then
# 31 days.  Add the -delete switch and you can clean
# up old content from a directory.
find /an/archive/directory -type f ! -newer reference_file

# You can easily script sql statements using this tool too
# consider that you need to just select yesterdays data
# for an archive:
START=$(datemath -d -1 -f '%Y-%m-%d 00:00:00')
FINISH=$(date +'%Y-%m-%d 00:00:00')
BACKUP_FILE=/absolute/path/to/put/backup/$START-backup.csv
# Now your SQL statement could be:
/usr/bin/psql -d postgres 
      -c "COPY (SELECT * FROM mysystem.data WHERE 
                mysystem.start_date >= '$START' AND 
                mysystem.finish_date < '$FINISH') 
          TO '$BACKUP_FILE';"
# Now that we've backed our data up, we can remove
# it from our database:
/usr/bin/psql -d postgres 
      -c "DELETE FROM mysystem.data  WHERE 
                mysystem.start_date >= '$START' AND 
                mysystem.finish_date < '$FINISH'"

Some Further Reading


dateblock: A Cron-Like Utility

dateblock allows us to mimic the functionality of sleep by blocking the process. The catch is dateblock blocks until a specific time instead of for a duration of time.

This very subtle difference can prove to be a very effective and powerful too, especially when you want to execute something at a specific time. Cron’s are fine for most scenarios, but they aren’t ideal in a clustered environment where this tool excels. In a clustered environment you may only want an action to occur on the node hosting the application, where a cron would require the action to fire on all nodes.

The python C++ library enables this tools usage even further since you can integrate it with your application.

Just like datemath is, dateblock is discussed in much more detail in my blog about it here.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref dateblock

# Get the python library too if you want it
yum install -y --enablerepo=nuxref python-dateblock

What You’ll Need To Know
Consider the following:

# block for 30 seconds...
sleep 30

# however dateblock -s 30 would block until the next 30th second
# of the minute is reached. Consider the time:
# date +'%Y-%m-%d %H:%M:%S' prints: 2014-11-14 22:14:16
# dateblock would block until 22:14:30 (just 14 seconds later)
dateblock -s 30

Dateblock works just like a crontab does and supports the crontab format too. It was written by me to specifically allow accuracy for time sensitive applications running. If you write your actions and commands in a script under a dateblock command, you can always guarantee your actions will be called at a precise time.

Since it supports cron entries you can feed it things like this:

# The following will unblock if the minute becomes
# divisible by 10.  so this would unblock at the
# :00, :10, :20, :30: 40: and :50 intervals
dateblock -n /10

# the above command could also be written like this:
dateblock -n 0,10,20,30,40,50

# You can mix and match switches to tweak the tool to
# always work in your favour.

A really nice switch that can help with your debugging and development is the –test (-t) switch. This will cause dateblock to ‘not’ block at all. Instead it will just spit out the current time, followed by the time it ‘would’ have unblocked at had you not put it into test mode. This is good for trying to tweak complicated arguments to get the tool to work in your favour.

# using the test switch, we can make sure we're going
# to unblock at time intervals we expect it to. In this
# example, i use the test switch and a cron formatted
# argument.  In this example, I'm asking it to unblock
# ever 2 days at the start of each day (midnight with
# respect to the systems time)
dateblock -t -c "0 0 0 /2"
Current Time : 2014-11-14 22:24:06 (Fri)
Block Until  : 2014-11-16 00:00:00 (Sun)

# Note: there is an extra 0 above that may through you
#       off.. in the normal cron world, the first
#       argument is the 'minute' entry.  Well since
#       dateblock offers high resolution, the first
#       entry is actually 'seconds', then minute, etc.

There are man pages for both of these tools. You’ll get a lot more detail of the power they offer. Alternatively, if you even remotely interested, check out e my blog entry on them.

The other cool thing is dateblock also has a python library I created.

# Import the module
from dateblock import dateblock

# Here is a similar example as above and we set
# block to false so we know how long were blocking until
unblock_dt = dateblock('0 0 0 /2', block=False)
print("Blocking until %s" % unblock_dt
              .strftime('%Y-%m-%d %H:%M:%S))
# by default, blocking is enabled, so now we can just
# block for the specified time
datetime.datetime('0 0 0 /2')

Some Further Reading


nmap (Network Mapper)

nmap is port scanner & network mapping tool. You can use it to find out all of the open ports a system has on it and you can also use it to see who is on your network. This tool is awesome, but it’s intentions can be easily abused. It’s common sense to scan your own network to make sure that only the essential ports are open to the outside world, but it’s not kind to scan a system to which you are not the owner of. Hence, this tool should be used to identify your systems weaknesses so that you can fix them; it should not be used to identify an others weakness. For this reason, if you do install nmap on your system, consider limiting permission so that only the root user has access to it.

# Assuming you've connected to my repository,
# just type the following:
yum install -y --enablerepo=nuxref-shared nmap

# Restrict its access (optional, but ideal)
chmod o-rwx /usr/bin/nmap

What You’ll Need To Know
Now you can do things like:

  • Scan your system to identify all open ports:
    # Assuming the server you are scanning is 192.168.1.10
    nmap 192.168.1.10
    
  • Scan an entire network to reveal everyone on it (including MAC Address information):
    # Assuming a class C network (24 masked bits) with
    # a network address of 192.168.1.0
    nmap -n -v -sP 192.168.1.0/24
    

Note: Don’t panic if your system appears to have more ports open then you had thought. Well at least don’t panic until you determine where you are running your network map test from. You see, if you run nmap on the same system you are testing against, then there your test might not prove accurate since firewalls do not normally deny any local traffic. The best test is to use a second access point to test against the first. Also, if your server sits on more then one network, make sure to test it from a different server on each network it connects to!

Some Further Reading

  • A blog with a ton of different examples identifying things you can do with this tool.

lsof (LiSt Open Files)

This tool is absolutely fantastic when it comes to preforming diagnostics or handling emergency scenarios. This command is only available to the root user; this is intentional due to the power it offers the system administrators.

I’m not hosting this tool simply because it ships with most Linux flavors (Fedora, CentOS, Red Hat, etc). Therefore, the following should haul it into your system if it’s not already installed:

# just type the following:
yum install -y lsof

# If this doesn't work, the rpm will be found
# on your DVD/CD containing your Linux Operating
# System.

What You’ll Need To Know
This tool when executed on it’s own (without any options) simply lists every open file, device, socket you have on one great big list. This can be useful when paired with grep if your looking for something specific.

# list everything opened
lsof

But it’s true power comes from it’s ability to filter through this list and fetch specific things for you.

# list all of the processes utilizing port 80 on your system:
lsof -i :80 -t

# or be more specific and list all of the processes accessing
# a specific service you are hosting on an IP listening on
# port 80:
# Note: the below assumes we're interested in those accessing
#       the ip address 192.168.0.1
lsof -i @192.168.0.1:80 -t

# If you pair this with the kill command, you can easily kick
# everyone off your server this way:
kill -9 $(lsof -i @192.168.0.1:80 -t)

But the tool is also great for even local administration; let’s say there is a user account on your system with the login id of bob.

# find out what bob is up to and what he's accessing on
# your system this way (you may need to pair this with
# grep since the output can be quite detailed
lsof -u bob

# Perhaps you just want to see all of the network activity
# bob is involved in. The following spits out a really
# easy to read list of all of the network related processes
# bob is dealing with.
lsof -a -u bob -i

# You can kill all of the TCP/IP processes bob is using with
# the following (if bob was violating the system, this might
# be a good course of action to take):
kill -9 $(lsof -t -u bob)

Perhaps you have a file on your system you use regularly, you can find out who is also using it with the following command:

# List who is accessing a file
lsof /path/to/a/file

Some Further Reading

  • lsof Survival Guide: A Stack Overflow post with some awesome tricks you can do with this tool.
  • More great lsof examples. This is someones blog who specifically wrote about this tool specifically. They provide many more documented examples of this tools usage here.

Credit

This blog took me a very long time to put together and test! If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

UFTP: Mass File Distribution Using Multicasting

Introduction

Multicasting has it’s pros and cons just like everything else. But it is often an overlooked solution to a common business problem which is: How do I transfer a file to multiple (subscribed) locations at the same time?
Most administrators or developers will come up with a solution that involves sending this file to each site using a common protocol such as SFTP, SCP, RSYNC, FTP, RCP, etc. There is no doubt that these solutions will work. However these solutions require you to send the file to each location individually. Hence, if you need to send a 10MB file to 100 remote locations, you’ll need 1GB (10MB x 100) of local network bandwidth to do it with.
Traditional Protocol File Transfers
A Multicasting solution saves you this effort and bandwidth by allowing you to send the file once and have all 100 sites collectively store it onto their systems at (relatively) the same time. Now, with respect to the diagram below (and the rest of this blog), I’m focusing entirely on the amazing efforts Dennis Bush put into UFTP. It is this tool that makes all of this possible. UFTP is one of those diamonds in the rough that I don’t think gets enough attention for the value it brings with it.
Multicast File Transfer using UFTP

Multicast in a Nutshell

A Multicast boils down to just being an IP address anywhere in the 224.0.0.0/4 network range. It uses a User Datagram Protocol to communicate with anyone who chooses to read and write data from this address. Routers can be configured to share this address across networks so that everyone may join in. Similar to a chat room on the internet where everyone joins together and anything they write, everyone else in the channel can see too.

Multicasting no doubt has it’s drawbacks. But I figure that it’s not worth dwelling on what you can’t do with the (multicast) protocol on it’s own since the UFTP tool this blog focuses on for mass file distribution has solved many (if not all) of these problems for us.

Why Multicasting, Who Uses it?

Multicasting saves bandwidth. Think of your Cable TV provider; when you change a channel, you’re actually just subscribing to a new multicast address along with the 100+ million other subscribing customers. Multicasting allows Cable TV companies to broadcast every channel at once, and those who are interested in a specific channel will receive it. Regardless of what channel you change to, there is no additional load sustained by the cable provider.

System administrators may not even know they’re using it when if they are managing a cluster. Most clusters use multicast as a way of passing their heartbeats to all other nodes in efforts to keep quorum.

Now, with respect to UFTP, if you visit it’s official website, you’ll see that the the Wall Street Journal used this tool in the past to send their developed newspapers to remote printing plants and other outlets across the United States.

UFTP in a Nutshell

UFTP is a File Transfer solution that wraps itself around the the multicast protocol as well as addressing the deficiencies that come with it. It is a well designed server/client application that allows for one to easily transfer files from 1 location to as many clients as you want in one shot. It can drastically save your company on bandwidth and has been around long enough to be deemed a reliable business decision. It’s worth noting that UFTP was written in C making it incredibly fast and lightweight on system resources. It works on all platforms but I’ll specifically focus on CentOS and the rpms I’ve packaged.

To use the software, you simply pass it a file and it looks after all the dirty work of guaranteeing it’s delivery to all of the clients subscribed to the address it broadcasts on. The author thought of everything when developing his tool, encryption of the data is always available to you as well if you prefer. He developed 3 main tools that you can manipulate to feed your data anywhere.
A Simple UFTP Environment

  • uftp (The Server – Docs): A simple command line tool that takes a file and broadcasts it to all of the listening clients. This is the exact reverse of the traditional ftp, sftp, scp, etc tools where they become the client and need to connect to a server to preform their tasks.
  • uftpd (The Client – Docs): A daemon that starts up and listens to a multicast address for new files being broadcasted by the server. Again, you’ll notice with Multicasting, roles are reversed from what you’d normally be used to. With the traditional protocols (FTP, SFTP, SCP, etc), they usually have daemons to host the server side of things not the client. You’ll want to run this daemon at all locations you wish to receive content broadcasted by the server (uftp).
  • uftpproxyd (The Proxy – Docs): The proxy allows you to tunnel your uftp multicast across a network that doesn’t support multicasting (such as the internet). This allows clients in other controlled networks (separated by one you can’t control) to additionally be part of your file distribution.

    The UFTP Proxy has 3 main modes it operates as:

    • Server Mode: This is used for pushing content upstream. This would effectively sit on the same server you call ‘uftp’ from. It listens just like any other client would and passes all information it receives to connected UFTP Proxies configured with the client mode.
    • Client Mode: This communicates with a UFTP Proxy configured for Server Mode and mimics the ‘uftp’ by broadcasting the same data to the local multicast address. This allows all local UFTP Clients to retrieve the file(s).
    • Response Mode: This is used to help take off some of the load of the server if there are many clients. Although the file is only being broadcasted once, there is a lot of handshaking that goes on at the start and end of the transmission to guarantee all data was delivered successfully. Depending on the different networks, their medium and reliability, a server may need some extra help with the handshaking if there are a lot of clients involved struggling to retrieve the data.

    Below shows an example of how the proxy can be utilized:
    A UFTP Proxy ConfigurationNote: Pinholes are used as a way to connect back to the UFTP Proxy Server effectively requiring no firewall changes to be made at each client site (only the server).

    **Note: It’s important to note a single Proxy Server can only be configured to connect to a single Proxy Client; it’s a 1 to 1 mapping. If you have multiple sites you need to connect to, you’ll need to set up an individual Proxy Server for each Proxy Client you need to serve.

    If you’re interested in the proxy portion of UFTP, you can read the official documentation about it found here.

Hand Over Everything

I wouldn’t have it any other way:

  • uftp-4.5.1-1.el6.nuxref.x86_64.rpm: The server is just the uftp tool and is really easy to use. This assumes you’ve got clients configured somewhere listening though!
    # just type uftp file.you.want.to.send
    uftp mytestfile
    
    # You can also send multiple files by just adding them to the end of
    # the string:
    uftp mytestfile1 mytestfile2 mytestfile3
    
    # I also wrote a small script that works the same way and sends stuff encrypted
    uftpe mytestfile1 mytestfile2 mytestfile3
    
  • uftp-client-4.5.1-1.el6.nuxref.x86_64.rpm: The client listens for data sent from the uftp server. You can use the RC Script i prepared to greatly simplify this tool:
    # as root; use the RC Script I wrote to make hosting the server
    # really easy
    service uftpd start
    

    Filtering is optional; if you don’t specify any, then by default there are no restrictions. Sometimes this is satisfactory (especially in closed or isolated networks). I attempted to simply UFTP’s built in server filtering for those who want to use it though. You see, not only can you encrypt the data you transmit from the server. But you can restrict the client to only accept connections from specific UFTP servers residing at a specific hosts with a specific server id. You can even go as far as only accepting servers with a specific fingerprint (created by their private key)

    # simply drop a configuration file in /etc/uftpd/servers.d 
    # Examples of accepted entries (taken from uftpd man page):
    # 0x11112222|192.168.1.101|66:1E:C9:1D:FC:99:DB:60:B0:1A:F0:8F:CA:F4:28:27:A6:BE:94:BC
    # 0x11113333|fe80::213:72ff:fed6:69ca
    #
    # You can have as many files as you want in this directory with as many entries in each
    # as you want.  If you add or remove new files, you'll need to restart the uftpd service
    # since it's only read in at the start.
    

    So by adding this bit of complexity, I know you are asking yourself:

    • Q: How do I know what my Server ID is?
      A: By default (using the rpm I’ve packaged) every server has an ID of 0x00000001 (decimal value of 1 – one) if you use the uftpe script I wrote (for encrypted transfers). To change your server id do the following:

      # Define a new id (other then 1)
      NEWID=2
      
      # Simply store this ID inside of the $HOME/.uftp config
      # file as it's HEX value:
      
      # Ensure the config directory exists
      [ ! -d $HOME/.uftp ] && mkdir -p  $HOME/.uftp
      
      # clear any old entry you may have set if you want:
      [ -f $HOME/.uftp/config ] && sed -i -e '/^UFTP_UID=/d' $HOME/.uftp/config
      
      # Set the new entry
      printf 'UFTP_UID=0x%.8xn' $NEWID >> $HOME/.uftp/config
      

      If however your just using the uftp tool on it’s own, it takes on the IP address of the host it’s running on (as it’s hex value) unless you explicitly specify -U 0x00000002 (or whatever ID you want it to assume). Here is a quick example of how you can convert an IP address to it’s hex value at the shell:

      # Define the address you want to convert
      IP_ADDR=192.168.1.128
      # Now convert it (don't forget the brackets!)
      (
         printf '0x'
         printf '%02X' $(echo "${IP_ADDR//./ }"); echo
      )
      # example above outputs: 0xC0A80180
      
    • Q: How do I know what my Server Fingerprint is?
      A: First off, uftp will not ever connect to this server with this option set if you choose ‘not’ to use the uftpe script i wrote or provide the necessary switches to enforce encryption. Setting a filter of your servers fingerprint is also a way of saying you only accept connections that are encrypted. This entry is completely optional. Your fingerprint ID is stored in $HOME/.uftp/uftp.key. Again this key only exists if your using the uftpe tool; otherwise the key is wherever you chose to store it. Fetch the id as follows (the below is intended to be called on the server where the uftp.key file exists):

      # fetch the details from the key:
      uftp_keymgt $HOME/.uftp/uftp.key
      

      Don’t panic if you don’t have a key; They are generated automatically when you first run the uftpe tool. The easiest way to pre-create it would just be to call uftpe by itself (without any parameters). Yes; it’ll spit an error telling you you didn’t provide it enough options. But the script will also generate you a key automatically too.

    I tried to think of everything; so log rotations and logging is already built in and included. You can locate them in /var/log/uftpd.log.

  • uftp-proxy-4.5.1-1.el6.nuxref.x86_64.rpm: the proxy service can bridge two networks that don’t support multicasting together. I haven’t spent to much time with this area since my environment hasn’t required me to. If you have any information to share about it; feel free and I can expand this area.
  • uftp-debuginfo-4.5.1-1.el6.nuxref.x86_64.rpm (Optional): This is only required if you are debugging this tool.
  • uftp-4.5.1-1.el6.nuxref.src.rpm (Optional): The source RPM for those interested in building the software for themselves.

Some Things To Consider:

When something is explained to be this easy, there is always a catch. I won’t lie, there are a few which means the UFTP solution may not be for everyone. They are as follows:

  • Multicasting isn’t enabled by most routers by default. If your recipients reside in a network you don’t manage, you’ll want to ask the local administrator to make sure their routers have (Level 2) multicasting enabled.
  • Multicasting can be a pain in the butt to troubleshoot; although newer routers won’t give you any grief, some of the older ones can fail to relay content correctly to clients who’ve also connected to the same multicast address. With negativity aside though, when it works, it works so great! My point is: you’ll need to make sure you’re using (networking) hardware that is relatively new (no older then 2010) where the firmware adequately supports Level 2 multicasting.
  • The UFTP Proxy attempts to resolve the problem of linking your networks together when separated by ones you don’t control. But consider that unless there are multiple recipients located on each network you connect your proxy too, you aren’t saving any bandwidth choosing this route.
  • Only uftp clients who are online will receive content sent by the uftp server. This can be a deal breaker for some especially if the product being delivered ‘MUST’ reach all of the clients. UFTP does not track who is online and who isn’t. It simply delivers content to those who are present at that time. It’s just like how you can’t watch a television show if you haven’t told your TV’s receiver what channel to be on.
  • If you’re using restrictive firewall settings (hopefully you are!), you’ll want to make sure multicasting is allowed into your client box with the following:
    iptables -A INPUT -m pkttype --pkt-type multicast -j ACCEPT

    It would be worth adding this to your /etc/sysconfig/iptables file as well.

Repository

Please note that all of these packages are also within my repository if that makes things easier for you and your deployment.

Sources

  • UFTP: Dennis Bush did a fantastic job with this tool. It is the key to making multicast file transmission powerful and reliable.
  • Multicasting: information in greater depth can be found here.

KeePassX – A Password Manager and Why You Need One

Introduction

I haven’t posted in a while and I figured I was due. I usually blog specifically for Linux users, but this really applies to anyone on any operating system (Apple, Microsoft, Android, etc).

I want to encourage the use of a password manager to anyone not already using one. There are a lot of them out there, but I’m specifically interested in focusing the attention on KeePassX. There are a number of reasons why and I’ll eventually get to most of them. But for starters KeePassX is completely free (GPLv2) and as I mentioned above, it works on just about every platform!

It’s sole purpose in life is to remember passwords for you so you don’t have to. It’s used to prevent you from using the same 1-3 passwords you’re using today for everything. Instead of using your birthday, engagement and/or pets name, you’ll use passwords like: JP83bW93wwrJsZ2x6tQy3aD6W or SLJUhoQWPBRtwTwmzsCxWXqOV. Again, I emphasis that the password manager does all the memorizing for you. Not only that, but it’ll easily generate you these passwords too.

KeePassX Interface

KeePassX Interface

Now before you roll your eyes and decide you don’t need such a thing, just here me out. Give me a chance to sell you the reasons why you seriously need to consider this. I’ll attempt to do this with respect to The 5 Stages of Loss and Grief. 🙂

Stage 1: Denial

What’s the point of a password manager? I never needed one before!
A password manager is kind of like social media was to us back in February of 2004. The ideology is the same: “You didn’t know you needed it until it was shown to you”. So trust me when I tell you that you need one now. The thing is, the internet is filled with people whose sole intent is to gain access to your identity and exploit it for their own personal gain. The passwords we’ve all used up to this date are too simple; I know this because we’ve all remembered/memorized them. Even the complicated ones we confidently memorized are no good.

Just Saying...

Food for Thought

Most of us just acknowledge there is risk, but we do nothing about it. We belive that if no one has figured out our password yet, they never will. Well the scary thing is: some of them might know our password(s) already. But these people don’t want to make that evident to us. Don’t expect someone to advertise themselves once they’ve gained access to something we had protected. Instead, you’re only going to find out once the damage has been done.

Stage 2: Anger

Why would I put all my passwords in one file? Something just sounds wrong with that statement.
You would do it for the same reason you trust a bank’s security with your money. The same reason you trust your wallet or purse with your personal identity. The same reason you go to work and secure all your belongings and pets behind in your home. We put all of our eggs into one basket every day, but we’re all smart about it. We all put trust in the decision that our belongings will remain in tact until we return to it. KeePassX works exactly the same way.

Both a Password and a Key File can be used to keep your password database safe.

Both a Password and a Key File can be used keep your password database safe.

The passwords that reside in your KeePassX database are encrypted, so no one will gain access to it unless you allow it to happen. The encryption means the password database is completely useless to anyone who has it without the keys required to access it. You can even go as far as to lock it using a keyfile (KeePassX will generate for you) as well as a password.

There is no question that if a hacker gains access to your password database and has all the keys he/she needs to unlock it, then you’ve defeated it’s purpose. Your passwords are only as secure as the conscious effort you take to keep them that way. If you keep the keys to your house under the welcome mat, or your car keys in the vehicle they pair with, it’s just a matter of time before you suffer a theft in that regard too.

Stage 3: Bargaining

Can’t I just change the passwords after I read about the compromising?
The answer is No and here is why:
People are attempting to exploit us ever day through phishing, viruses, spyware, false advertising, and even through everyday tasks we didn’t even know we were a victim of. Take Sony and eBay for example. Both are very reputable companies with an entire team of security experts working with them every day, yet they still fell prey when their systems were compromised by a hacker. As a result, it’s the clients and customers that become the victims too (all of us). If you used the same password on these sites as you do everywhere else, then you’re already at risk for identity theft. The point is, sh*t happens and sometimes it’s just out of our control.

We can’t expect every organization to be like eBay and Sony either. Not all of them will alert their customers of the threat they faced or the security breach they sustained. The few that do go public may have been exploited long before they did. Hence, our personal information was at risk last week when they told us today. There are even some companies who don’t want to admit to any privacy invasions to protect their own reputation. Then there are the companies that might not even know they’ve already been compromised. So if they don’t tell us; how will we ever know?

Therefore, consider taking the extra time to change every password on every site you visit once you get KeePassX installed.

Stage 4: Depression

I don’t want to change my password everywhere.
I don’t want the burden of a another application just to manage my passwords.
I was stuck at this stage for a while. It takes a while to change passwords everywhere. But after it’s done, you have to remember that this program does absolutely everything in its power to make your life easy from this point on. You only need to add everything once; so the burden doesn’t linger.

How can such complicated password make my life easier?

Ctrl+B and Ctrl+C are all you ever need to use once KeePassX is setup.

Ctrl+B and Ctrl+C are all you ever need to use once KeePassX is setup.

Your life ‘will’ be simpler with KeePassX because your every day tools are now accessible by just a few keystrokes:

  1. Press ctrl-b in Password Manager: Copy the username into the clipboard
  2. Press ctrl-v in Web Browser or App: Paste the username (into the username field)
  3. Press ctrl-c in Password Manager: Copy the password into the clipboard
  4. Press ctrl-v in Web Browser or App: Paste the password (into the password field)

That’s it; don’t worry about the password (you just copied) being left in the clipboard. KeePassX looks after clearing the clipboard after a few idle seconds elapse that you configure (the default is 20 seconds). The point of me explaining this is: instead of memorizing your secret passwords you use everywhere, you only need to remember the 4 key combinations identified above. The same 4 keystrokes are used for all sites/apps you ever use/visit from this point forward. Not to mention that each password is a unique from the other and unguessable by any hacker. I can’t stress enough that the effort level decreases with a password manager. At the same time your online privacy is more secure than ever.

Stage 5: Acceptance

What about my Phone? What about my other computer? Can access my password file on other systems?
Yes! This is one of the most amazing features of KeePassX! People even place their password file on locations such as DropBox or their Google Drive so they can access it between the systems. It’s compatible with Microsoft Windows, Mac OS (including Apple iOS), Android, and Linux (of course!).

So Where Do I Get It?

Microsoft Windows and Mac OS users can just download it directly from the download area at the official KeePassX website here.

If you’re using an Android device then you can find it here on the Google Play Store.

If you’re using an Apple device then you can find it here on the iTunes Store.

Linux users can download it from pkgs.org. It’s also available from the EPEL Repositories.

For the CentOS and/or Red Hat users: if for whatever reason the links above become unavailable or you want a fast approach: KeePassX can be retrieved from the repository I’m hosting.

Install KeePassX in CentOS and/or Red Hat 6 :

# Make sure you hook up with my repository first:
# Visit : http://nuxref.com/nuxref-repository/

################################################################
# Install our required products
################################################################
yum install -y --enablerepo=nuxref keepassx
# You're Done!

KeepassX Tips

KeePassX Password Generator

KeePassX Password Generator will keep your passwords diverse and complicated.

  • For obvious reasons, password protect your password database. This is the one password you ‘will’ need to remember. So I encourage you to use a new original one, but don’t forget it!
  • Consider using a keyfile as well as the password for securing your password database. The nice thing about the keyfile is you can move it with you; keep it on a usb drive that is kept with you or in another location. This way if someone ever did get a hold of your password database, they can’t do anything without the key file even if they know your password.
  • Configure KeyPassX to lock itself after it’s been idle or minimized. By default it won’t, but it’s a simple option that will allow you to walk away from your desk and know that no one is snooping where they shouldn’t be.
  • Log onto your remote sites and consider changing your passwords every now and then. Even if it’s only once a year. Just changing your passwords annually is still better than never changing them at all! You can optionally configure KeePassX to remind you to change your passwords after they get to a certain age. Just using the built in password generator will greatly simplify your life and keep your passwords complex.

Speaking of Passwords…

Consider that your neighbors (or even someone you don’t know) could be using your wireless network while you’re at work. They could be using it even when your around. Have you been connecting to your wireless router lately to see if you can account for everyone connected to it? Now would be a good time to change this password too!

Pay attention to the sites you use and make sure they use some form of login encryption. For example, logging into a website that isn’t secure means anyone can easily extract your username password combination without your knowledge.

Privacy Compromised

Just do a Google search for +breach +password +hacked to get an idea of how many companies are getting hacked constantly and how easy it is for someone to figure out your password and re-use it elsewhere.

Final Notes
Sure, Password Managers aren’t for everyone; don’t worry, I get that. But if you truly want to prepare yourself and prevent having to deal with unnecessary online fraud, or identity theft… If you truly wasn’t to avoid venturing through The 5 Stages of Loss and Grief which follows these awful crimes, then you should consider protecting yourself now. Take the plunge and get a password manager and diversify your passwords.

It’s better to be safe than sorry.

Credit

If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Sources

Here are some other links you may find useful: