Tag Archives: Linux

The Perfect Home Assistant CentOS 8 Installation

Introduction

If you have a dedicated server that you want to set up Home Assistant on and would like to run it natively (not in a container), then this is the tutorial for you. This tutorial focuses on using CentOS 8 because I am personally familiar with RedHat based products. But those who chose to use Ubuntu, Debian, Alpine, Arch, etc should be able to benefit from a good chunk of the information explained here too.

Minimum Requirements

While Home Assistant works awesome on a Raspberry Pi for simple things, it can start to get sluggish and feel under-powered over time as you get addicted like the rest of us and continue to add automations to it. It specifically takes a hard it once you start adding automations that involve video and get fancy with others.

nuc6i5syh
Intel Nuc Kit nuc6i5syh
Thus, if you’re a hobbyist who plans on continuing to evolve your homes automation over time, a Raspberry Pi is not good enough. A small PC (and this tutorial of course πŸ˜‰ ) is the best way to go.

I am personally using a very old (outdated) model of an Intel Nuc that works great. But even if you just have an old PC lying around in the corner, that would be a fantastic choice too! Despite the official requirements identified here, for the most optimal experience (in my opinion), your Home Assistant PC should have:

Type My Suggested Minimum Requirement
RAM 8GB
CPU >= Intel i5
Network >=100Mbit/s
It’s strongly recommended that the server you choose to set up Home Assistant utilize a physical network cable and not WiFi.
Storage >=32GB
Storage is only really important if you plan on installing home surveillance (specifically video cameras). If you do, then target your storage to be at least 512GB or more instead. This will allow you to record with them and save the content locally (should you choose to do this).

Here is the breakdown of where the storage all goes (with respect to this tutorial):

Path Size Details
/etc/homeassistant/ 1GB Your configuration files are small (less then 256KB), however Home Assistant maintains what is called a Recorder which tracks itself in an SQLite database. This SQLite database tends to grow over time in size (which you can control). This is also the location of your homeassistant.log file which can also grow in size as well. By giving yourself (at least) 1GB here, you just provide enough room for future growth that will never interrupt your setup over time. More on the contents of this directory is discussed near the end of this blog.
/opt/homeassistant/ 2GB The entire installation of Home Assistant will sit here once you’ve completed this tutorial. The actual installation (with most bells and whistles) is just under 1GB. 2GB is just a suggested size because it will leave you with plenty of room for future growth.
/ 6GB Your root filesystem and all the core directories that go with it such as /boot, /tmp, /home, /usr, /var, … etc.
6GB is more than enough to host the Linux operating system and all of it’s needs while still leaving you ample space for growth.
/opt/homeassistant/video/ >128GB I’m just throwing this in here… Those who don’t have video recording cameras at their home (inside and out) can ignore this. But consider at LEAST 32GB per camera depending on how much you want to record. You may also want to plan for an expansion down the road too; I said >=512GB earlier and I meant it. Storage is cheap these days, so plan properly at the start.

Download the latest copy of CentOS 8 and stick it on a spare USB Drive you have and/or burn it to a DVD so that you can install it onto your PC.

The Base Home Assistant Setup

It is assumed at this point that you’ve set up your server or already had one running. Let’s set up a working environment that Home Assistant can work in. It’s important to have root or sudo privileges prior to running the commands below in each section.

General Server Preparation

The safest way to run Home Assistant is with it’s own dedicated user and permissions. The following will set up a working environment for us to build further onto:

# Again.. our basic layout will be:
# User: homeassistant
# HA Config: /etc/homeassistant
# HA Core: /opt/homeassistant

# Prepare our user:
sudo useradd -rm homeassistant \
   --home-dir /opt/homeassistant

# The dialout group allows us to access our external USB
# devices such as the Aeotec Z-Stick Series 5 (which is
# what I use). We want to add our user to this group:
sudo usermod -aG dialout homeassistant

# Prepare our configuration directory
sudo mkdir /etc/homeassistant

# Adjust our permissions
sudo chown -R homeassistant:homeassistant \
   /opt/homeassistant /etc/homeassistant

# Some of the packages later installed with pip3
# require that they are compiled in your environment
# (or it will not work) so the following is also
# required:
sudo dnf install -y gcc make gcc-c++ systemd-devel \
    unzip tar

Python 3.8 Setup

We need to setup Python 3.8 into our CentOS environment because it is the minimum requirement for Home Assistant to work. So basically we have 2 options here: Take the most stable version of v3.8, or take the one manged by CentOS which is sometimes outdated. Either way, I’ve explained both below:

  1. Option 1: The CentOS Maintained Version:
    # Simply install the module
    sudo dnf module -y install python38 python38-devel
    
    # That's it... you're done :)
    
  2. Option 2: The Latest Stable Version:
    sudo dnf install -y bzip2-devel expat-devel gdbm-devel \
        libuuid-devel libffi-devel\
        ncurses-devel openssl-devel readline-devel \
        sqlite-devel tk-devel xz-devel zlib-devel wget \
        gcc make gcc-c++ tar
     
    # At the time this blog was written, the current
    # version was v3.8.5:
    VERSION=3.8.5
    
    #
    # Acquire
    #
    wget https://www.python.org/ftp/python/${VERSION}/Python-${VERSION}.tgz
    tar -xf Python-${VERSION}.tgz
     
    #
    # Configure
    #
    cd Python-${VERSION}
    ./configure --enable-optimizations
    
    #
    # Build
    #
    
    # This step can take a while - so be patient!
    make -j 4
    
    #
    # Install
    #
    
    # This is very safe and will not over-write
    # existing Python that ships with CentOS
    sudo make altinstall
    

Regardless of which route you chose to take, you can test the version out to see if you were successful:

# Test the version out:
python3.8 -V

Surveillance Video Camera Setup

Run the following commands if you have or plan on having surveillance cameras installed into your environment. Feel free to run this at a later time if you want. Basically you need a copy of ffmpeg available if you plan on accessing your camera stream in Home Assistant.

sudo dnf install -y epel-release dnf-utils
sudo yum-config-manager --set-enabled PowerTools
sudo yum-config-manager \
   --add-repo=https://negativo17.org/repos/epel-multimedia.repo
sudo dnf install -y ffmpeg

Home Assistant Installation

Assuming you’ve followed the sections Python 3.8 Setup and General Server Preparation above, we can now install Home Assistant:

# First we switch to our new homeassistant user we created:
sudo -u homeassistant -H -s
 
# Optional SSH Keygen Setup
[ ! -f ~/.ssh/id_rsa.pub ] && \
   ssh-keygen -f ~/.ssh/id_rsa -C "Home Assistant" -q -N ""

# Change to our home directory (if we're not there already)
cd /opt/homeassistant

# Prepare our virtual environment
python3.8 -m venv .

# It's always VERY important you activate your
# environment before you start running pip3
# commands
. ./bin/activate

# Perform our installation:
pip3 install homeassistant home-assistant-frontend \
     homeassistant-pyozw colorlog flask-sqlalchemy

# Return back to our root user
exit

Finally we must create an System D Startup file to make it easy to start/stop our instance of Home Assistant:

# the following will drop us in a startup script we can use with our environment
sudo bash <<EOF
cat << _EOF > /usr/lib/systemd/system/home-assistant.service
[Unit]
Description=Home Assistant
After=network-online.target
 
[Service]
Type=simple
User=homeassistant
ExecStart=/opt/homeassistant/bin/hass -c "/etc/homeassistant"
WorkingDirectory=/opt/homeassistant
PrivateTmp=true
Restart=on-failure
 
[Install]
WantedBy=multi-user.target
_EOF
EOF

# Pick up our new configuration
sudo systemctl daemon-reload

# Make it so Home Assistant starts even after a
# server reboot
sudo systemctl enable home-assistant

# Let's start it up now
sudo systemctl start home-assistant

The final piece of the puzzle is to expose port 8123 to our network so that we can access our instance of Home Assistant:

sudo bash <<EOF
cat << _EOF > /usr/lib/firewalld/services/home-assistant.xml
<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>Home Assistant</short>
  <description>Home Assistant Web Portal</description>
  <port protocol="tcp" port="8123"/>
</service>
_EOF
EOF

# Reload our firewall so it can find our recently
# created configuration file (defined above)
sudo firewall-cmd --reload
 
# Now we'll add our new firewall service
sudo firewall-cmd --zone=public \
   --add-service=home-assistant --permanent

# once again reload our firewall to open our newly
# declared port:
sudo firewall-cmd --reload

Finalizing Your Installation

Home Assistant Initial Login PageYou can now visit your new Home Assistant server at http://ip.to.your.server:8123 and start setting up your account for the first time.

When you first access the page you will be able to create the first system user (also the administrator account) that you can build automations with.

Home Assistant Upgrades

You’ll find that Home Assistant is rapidly expanding and always fixing issues and adding more cool features you can play with. Upgrading Home Assistant is as easy as this:

# Switch to our home assistant user:
sudo -u homeassistant -H -s

# Change to our home directory (if we're not there already)
cd /opt/homeassistant

# It's always VERY important you activate your
# environment before you start running pip3
# commands
. ./bin/activate
 
# Switch to our homeassistant user
pip3 install --upgrade homeassistant

# Return to our user with sudoer's permission
exit

# Restart Home Assistant
sudo systemctl restart home-assistant

A Quick Overview

At this point you should be all set up with a running copy of Home Assistant to play with. Here are some general files and directories that will begin appearing in your server’s filesystem in the /etc/homeassistant/ directory that you may want to know about:

homeassistant.log

All of the logs associated with Home Assistant will show up here. This is a fantastic location to begin your troubleshooting should you run into problems.

configuration.yaml

This is the core configuration that will drive your entire Home Assistant application from this point forward. Any changes to this file will require you to restart Home Assistant for the changes to take effect.

home-assistant_v2.db

This is the Home Assistant Recorder. From within your Home Assistant Dashboard, you can click on History button off of the dashboard to see the contents.

/etc/homeassistant/home-assistant_v2.db example

secrets.yaml

This is a simple file that allows you to map key/value pairs. The idea is to keep all of your passwords and/or tokens that you otherwise want to keep from prying eyes. This file should be given very strict permissions:

# protect this file if you intend to use it
chmod 600 /etc/homeassistant/secrets.yaml

For example, you might use Apprise to send you an email notification. In order to send an email Apprise needs your personal email information. As an example, in your configuration.yaml file you might do this:

# /etc/homeassistant/configuration.yaml
# ...
notify:
  name: apprise
  platform: apprise
  url: !secret apprise_url

Then in your secrets.yaml file you now need an entry for a new keyword you created called apprise_url. Your entry in the secrets.yaml file might look like this:

# /etc/homeassistant/secrets.yaml
# Define your Apprise details in a secure location:
apprise_url: mailto://myuser:mypassword@gmail.com

More details on how secrets work can be found here in the Home Assistant documentation. Also, check out the Apprise wiki to see all of the different services you can notify with it.

automations.yaml

This is where you can begin constructing your own Home Assistant automations once you’ve added a few integrations into Home Assistant.

An example of an automation you might have set up could send us an email using Apprise at sunset:

# /etc/homeassistant/automations.yaml
#
# Utilize Apprise to send a notification
#
alias: "[Interactive] - Sunset Apprise Notice"
trigger:
  platform: sun
  event: sunset

action:
  service: notify.apprise
  data:
    title: "Good evening"
    message: "This sun is setting."

Configuration Backup

No setup is complete without regular backups taking place. The following is just a basic configuration backup setup you can use to get you started:

# For Backups (run these commands as the homeassistant user):
sudo mkdir -p /opt/homeassistant/backups
sudo chown homeassistant.homeassistant /opt/homeassistant/backups

# This is a simple script to get you started:
sudo bash <<EOF
cat << _EOF > /opt/homeassistant/bin/backup.sh
#!/bin/sh
# Create a backup of Home Assistant
TARGET=/opt/homeassistant/backups
SOURCE=/etc/homeassistant
 
[ ! -d "\$TARGET" ] && /usr/bin/mkdir -p "\$TARGET"
[ ! -d "\$TARGET" ] && exit 1
 
tar cfz "\$TARGET/\$(date +'%Y.%m.%d')-config.tgz" -C \$SOURCE . 
RET=\$?
 
# Tidy; Keep X days worth
find -L \$TARGET/ -mindepth 1 -maxdepth 1 -name "*.config.tgz" -mtime +120 -delete
exit \$RET
_EOF
EOF

# Permissions to be executable
sudo chmod 775 /opt/homeassistant/bin/backup.sh

# Prepare ourselves a cron job that will run our backup every week
sudo bash <<EOF
cat << _EOF > /etc/cron.d/homeassistant_backup
0 0 * * 0 homeassistant /opt/homeassistant/bin/backup.sh &>/dev/null
_EOF
EOF

Credit

This blog took me a very (,very) long time to put together and test! If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask. Alternatively, I certainly won’t turn down a coffee if you wish to buy me one! πŸ™‚

Sources

Integrate Apprise into Nagios for More Notification Support

Introduction

Apprise is an open source tool that allows you to send a notification through a wide range of messaging services out there (such as Discord, Slack, Telegram, Microsoft Teams, etc). Well when you combine this with Nagios, you open it up to a much larger scope then simply emailing on an alert.

With Apprise you can configure Nagios to text your mobile phone using Amazon’s Web Service, or notify your devops team on Slack and/or Microsoft Teams. You can even trigger an IFTTT event. But it doesn’t just stop there, Apprise already supports over 35+ notification services today (and is always expanding) which means Nagios could leverage all of this too. This blog will explain how you can set up your instance of Nagios to notify more end points then just email.

The Installation

I’m going to presume you have a copy of Nagios already installed. If you don’t you can check out my blog here on how to set up your own copy with CentOS 7. Those who are not using CentOS are certainly not out of luck though, there are lots of blogs out there to get you started.

This blog will assume you have root privileges or have sudoers privileges.

Apprise can be easily added to your system through pip:

# Install Apprise onto the system currently also hosting Nagios
sudo pip install apprise

Configure Nagios

The Nagios configuration files can vary in their location depending on what Linux distribution you’re using. I’m going to just refer to some standard paths used by the stuff I host here (for CentOS/RedHat).

Step 1: Nagios Import Directory

If you’re using the Nuxref RPMs, then you can skip this step and move to the next as you’ll already be configured for this. Those using another distribution will want to update their nagios.cfg to point to a directory we can use to drop in and remove configuration from. The file is presumably going to be located as: /etc/nagios/nagios.cfg):

# Place this anywhere in /etc/nagios/nagios.cfg
# preferably put it near the bottom of the file.

# Definitions for global configuration directory
cfg_dir=/etc/nagios/conf.d

Now make sure this directory exists because this is where we’ll place our new apprise configuration:

# Ensure our global include directory exists that we
# just defined in our nagios.cfg file:
mkdir -p /etc/nagios/conf.d

# Place a dummy file in here so that Nagios doesn't
# throw any errors (as it isn't a fan of include directories
# without configuration files in it).
touch /etc/nagios/conf.d/dummy.cfg

Step 2: Apprise/Nagios Integration

Now we need to let Nagios know about Apprise. We’ll do this by creating the following files called /etc/nagios/conf.d/apprise.cfg

#
# Apprise to Nagios Configuration File
# Place this file as /etc/nagios/conf.d/apprise.cfg
#
# 'notify-host-by-apprise' command definition
define command{
   command_name   notify-host-by-apprise
   command_line   /usr/bin/printf "%b" "- *Notification Type*: $NOTIFICATIONTYPE$\n- *Host*: $HOSTNAME$\n- *State*: \n- *Address*: $HOSTADDRESS$\n- *Info*: $HOSTOUTPUT$\n\n- *Date/Time*: $LONGDATETIME$\n" | /usr/bin/apprise -c /etc/nagios/apprise.yml -n "$HOSTSTATE$" -g "$NOTIFICATIONTYPE$" -t "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **"
}
 
# 'notify-service-by-apprise' command definition
define command{
   command_name   notify-service-by-apprise
   command_line   /usr/bin/printf "%b" "*Notification Type*: $NOTIFICATIONTYPE$\n- *Service*: $SERVICEDESC$\n- *Host*: $HOSTALIAS$\n- *Address*: $HOSTADDRESS$\n- *State*: $SERVICESTATE$\n- *Date/Time*: $LONGDATETIME$\n\n*Additional Info*:\n$SERVICEOUTPUT$\n" | /usr/bin/apprise -c /etc/nagios/apprise.yml -n "$HOSTSTATE$" -g "$NOTIFICATIONTYPE$" -t "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **"
}

# Register our contact template that we can reference
define contact{
  ; The name of this contact template
  name                             apprise-contact

  ; service notifications can be sent anytime
  service_notification_period      24x7

  ; host notifications can be sent anytime
  host_notification_period         24x7

  ; send notifications for all service states, flapping events,
  ; and scheduled downtime events
  service_notification_options     w,u,c,r,f,s

  ; send notifications for all host states, flapping events,
  ; and scheduled downtime events
  host_notification_options        d,u,r,f,s

  ; send service notifications via email
  service_notification_commands    notify-service-by-apprise

  ; send host notifications via email
  host_notification_commands       notify-host-by-apprise

  ; Don't register this as it is just a template for future
  ; references by contacts who wish to use the apprise plugin
  register                         0
}

Now for every contact we set up going forward, we can point it to use Apprise. By default Nagios usually provides us a contact.cfg file that contains the generic user nagiosadmin. For those using my packaging, you can find this file at /etc/nagios/objects/contacts.cfg; you’ll want to change it to looks like this:

define contact{
  ; Short name of (Nagios) user
  contact_name    nagiosadmin

  ; This next line used to read generic-contact; but we want to switch it
  ; over to our new Apprise based one:
  use             apprise-contact

  ; Full name of user
  alias           Nagios Admin

  ; not important if using apprise-contact (defined above)
  email           nagios@localhost
}

Before you advance to the next step, you’ll want to run a test flight check on your configuration and make sure it validates okay.

# Perform a flight check on our new configuration (as root)
sudo nagios -v /etc/nagios/nagios.cfg

If you get any errors, you should revisit the first part of this blog and try to iron them out before continuing. If everything is error free, then the next step is to reload our instance of Nagios (if it’s running) so it can re-read this configuration. This can be done with the command:

# You will need to be root to do this; send a SIGHUP
# to all instances of nagios running in memory:
sudo killall -HUP nagios

Step 3: Apprise Configuration

Now we need to prepare our Apprise configuration (/etc/nagios/apprise.yml) and fill it with the notification services we want listen for and who we want to pass it to.

We can associate with Nagios notifications types passed to us through tags. Nagios will pass these along one of the following $NOTIFICATIONTYPE$ when an event occurs; these are:

  • PROBLEM: There was an issue with one of the checks.
  • RECOVERY: The issue previously set has been cleared.
  • ACKNOWLEDGEMENT: An outstanding issue has been acknowledged.
  • FLAPPINGSTART: Flapping is a state where a service has a PROBLEM associated with it and then moments later has a RECOVERY. This is the state called FLAPPING. When this process occurs too many times in a row, this alert gets set.
  • FLAPPINGSTOP: The service that was previously FLAPPING is no longer doing so.
  • FLAPPINGDISABLED: Someone just disabled FLAPPING for this service/host.
  • DOWNTIMESTART: The scheduled downtime for this service/host has begun.
  • DOWNTIMEEND: The scheduled downtime is over.
  • DOWNTIMECANCELLED: Someone just cancelled the scheduled downtime for this service/host.

Knowing the above notification types that we’ll receive, here is what an Apprise configuration file located at /etc/nagios/apprise.yml might look like:

# This file should be placed in /etc/nagios/apprise.yml

# NOTE: THIS IS JUST AN EXAMPLE CONFIGURATION FILE. YOU WILL WANT
#       TO CUSTOMIZE YOUR OWN WITH THE SERVICE(S) OF YOUR CHOICE
#       VISIT https://github.com/caronc/apprise TO SEE WHAT IS
#       AVAILABLE AND HOW THEY WORK.

# Identify all of the global notification types we want to flag on.
tag:
  - PROBLEM
  - RECOVERY
  - FLAPPINGSTART
  - FLAPPINGSTOP

# Now we want to define our Apprise URLS; you'll want to visit
# https://github.com/caronc/apprise to see all of the supported
# services and how to build their URLs.
urls:

  # Maybe we want to notify a custom service we're hosting to
  # monitor and track Nagios status; Check out the following 
  # for more details https://github.com/caronc/apprise/wiki/Notify_json

  - json://localhost

  # Maybe we want to notify a Slack channel; more details on this
  # are here: https://github.com/caronc/apprise/wiki/Notify_slack
  - slack://T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F/#nuxref

  # the Apprise YAML configuration is quite powerful, the
  # following prepares the email URL and sends an email to each
  # user identified below:
  - email://user:password@gmail.com
      - to: george@example.com
      - to: admin@example.com

  # More details on the emails can be found here:
  # https://github.com/caronc/apprise/wiki/Notify_email

  # We can also individually disperse the tags in the same config
  # file.  The below tags will override the globals defined above.
  # A use case for this would be that maybe we just want to
  # send certain notification types to say... the DevOps team:
  - email://user:password@gmail.com
      - to: devops@example.com
        tag: DOWNTIMESTART, DOWNTIMEEND, DOWNTIMECANCELLED

Once you’re file is all ready, be sure this file is readable by Nagios (to keep it away from prying eyes), but otherwise you’re all set and ready to go!

# Here is what one might do to protect this apprise configuration
chmod 640 /etc/nagios/apprise.yml
chown nagios.root /etc/nagios/apprise.yml

Verification

To be sure everything works, you may want to just test that you got all of your configuration right You can test this using manually as follows:

# Test our configuration with apprise using the PROBLEM tag
# -vvv for some verbose debugging in-case we need it.
apprise -c /etc/nagios/apprise.yml \
    -n CRITICAL -g PROBLEM \
    -vvv \
    -t "A Test Title" \
    -b "a Test Body"

Here is a screenshot of a test error displayed on gitter.im that was sent by Nagios using Apprise:
Gitter Example

Sources

Pan: A Useful NewsReader for Linux

Introduction

PAN is a newsreader that has been around for ages. It allows you to sift through the massive clutter that Usenet has become through its really fast interface loaded with tons of features!

It’s development died off way back in 2012, but recently it’s development has picked right back up again. Not only is this product feature rich and open source, but it’s written purely in C++ which makes it incredibly light weight (thus very, very fast). Some of the subtle product enhancements this product has seen in the past few months make it worthy to be in the spot light again.

So What Can It Do?

  • Header Caching: Tell it the group(s) you want and how much of it you want to see and it will download the headers it retrieves to a local cache file. This is awesome because now you can sift through this content offline.
    Cache Headers
    Cache Headers
  • Header Scoring: You can flag key aspects of articles with a score. By default every header retrieved has a score of zero (0) unless you start dabbling in this area.

    Anything that scores less than (or equal to) -9999 can be configured to not list itself at all. Some well set scores can greatly clean up your ability to locate content in groups.

    You can score content higher and/or lower based on the posts author, subject, size, age, etc. You can even apply scoring through regular expressions too!

    Scoring is very powerful when used properly! I’ll talk about it again a bit later in this blog once you’ve gotten set up. But if we were to apply scoring to the previous screenshot (above), it might look like this (all garbage cleaned up and content prioritized with color coding too):

    Header Scoring
    Header Scoring
  • Multiple Server Support: Got a block account? No problem, you can add it as a secondary server and only pull from it if the Primary one is unavailable.
  • NZB-File Support: The treasure maps of Usenet can be loaded into Pan too and downloaded through it. True automation of these come through systems like NZBGet and SABnzbd, but it’s still worth knowing that not only is this a newsreader, but it can pass as a downloader as well!
  • Concurrent Connections: Like any great browser/downloader of any system; files are retrieve concurrently. This means that you can just keep browsing and tagging content of interest seamlessly without interruptions.
  • Header Compression Support: One of the new enhancements surfaced with the new development of this project. This makes a world of difference when retrieving hundreds of thousands of headers from a Usenet group. Enabling this feature along (if your Usenet provider supports it) will greatly reduce wait times!

Pan’s Disabled Features

The features page on PANs website explains about a parent company (called ChimPanXi) that tries to sell this free product with added functionality. I guess the deal they have with the developers is to just disable a few features so that they can be re-enabled them the paid version (purely speculation)?

But since the (Pan) code is open source, the options are right there in front of us but just disabled. Quite honestly… of all this disabled functionality, only one is truly worth pointing out: Pan restricts you to just 4 allowable concurrent connections to your Usenet provider at a time. Here is a small patch I created which increases this number to 99. The build I provide in this blog already has this patch applied. Here are the rest of the missing features (with some of my comments as well); maybe some might see value in the others?

Pans Missing Features
Pans Missing Features

The Goods

For those hooked up to my repository are already set, just type the following:

# install the new version of Pan
yum install pan --enablerepo=nuxref

You can also reference this table too for direct links:

Package Download Description
pan el7.rpm, fc22.rpm, fc23.rpm, fc24.rpm, fc25.rpm The Newsreader: This is the program that this blog focuses on.

Note: The source rpm can be obtained here which builds everything you see in the table above. It’s not required for the application to run, but might be useful for developers or those who want to inspect how I put the package together.

It’s also worth noting (again) that this build includes a small patch to increase the maximum allowable number of concurrent connections from 4 to 99.

Securing Your Connection

There is very little security built into Pan from a connection point of view. What little security is (normally) in place is built using GnuTLS. GnuTLS has a history of not keeping up with the security exploits and vulnerabilities that surface with encryption libraries. It doesn’t make it unsafe; it just doesn’t make it as reliable as it’s competition (OpenSSL and Crypto). For this reason the packages I provide are intentionally not built against it (GnuTLS).

It’s really not a problem at the end of the day because there are other ways of securing this connection (properly). The way I use (and recommend) is through Stunnel.

Stunnel allows you to take an unencrypted input (from Pan) and connect it to a secure connected one (at your Usenet provider). The best thing about stunnel is that it links to your (OpenSSL) shared system libraries libssl.so and libcrypto.so which are actively maintained and patched! Basically what I’m saying is by attaching Pan to Stunnel: you get the feature rich usage of Pan and the ongoing (reliable) security of OpenSSL.

The following will get you set up with stunnel; you’ll want to be root before running the command below:

# Install stunnel (if it's not installed already)
# you'll need to be connected to either EPEL or NuxRef for this
# to work:
yum install stunnel

You can also reference this table too for direct links:

Package Download Description
stunnel el7.rpm, fc22.rpm, fc23.rpm, fc24.rpm, fc25.rpm Secure Tunnel: for data encryption.

Note: This RPM is not required by PAN to run correctly. It does however offer you a safer and more secure method of encrypting your communication to (and from) your NNTP Server.
# You must have root permissions when setting up
# stunnel

# Create relay bound to local server only (semi-colons are for
# comments):
cat << _EOF > /etc/stunnel/stunnel.conf
; Use it for client mode
; This is the pass through mode you need to encrypted
; your NNTP traffic:
client = yes
 
[nntp]
;
; --- IN ---
;
; local port to listen on (on this PC)
; You will configure PAN to connect here:
accept = 127.0.0.1:119

;
; --- OUT ---
;
; The Remote Usenet Server's (encrypted) connection to use:
; In this example, I'm just pointing to Astraweb, but you
; can provide any Usenet server you wish here. Just be sure
; to point it to their secure transport point!
connect = ssl.astraweb.com:563
_EOF

# This line below is useless, but it allows you revisit this blog
# entry and continue and copy and paste these instructions at a later
# time. The line removes any previous entries set to prevent the
# creation of duplicate entries  in your startup file at another time
# It's harmless to run at any point:
sed -i -e '/bin\/stunnel/d' /etc/rc.d/rc.local

# Configure stunnel to start after each boot
echo "# Start /usr/bin/stunnel on boot each time:" >> /etc/rc.d/rc.local
echo "/usr/bin/stunnel" >> /etc/rc.d/rc.local

# By default stunnel is configured to read 
# it's configuration from /etc/stunnel/stunnel.conf
# on startup:
stunnel

The next step is to update your PAN server configuration to point to your local server (localhost or 127.0.0.1) instead of the remote one you’re accessing. Make sure to set the port to 119 too like so:

Stunnel Pan Configuration
Stunnel Pan Configuration

You’ll provide the same username and password you would have otherwise provided to your Usenet provider.

The end result is a secure connection between you and your Usenet provider like so:
Pan Setup With Stunnel

Scoring

Scoring articles can greatly ease your life when looking through all of the headers in front of you; it’s great for:

  • Eliminating SPAM
  • Filtering out potential malicious content (such as Trojans and Viruses)
  • Increasing the visibility of items of interest
  • Locating Authors of interest with ease

All scores can be optionally associated with a time limit too. When the limit expires, so does the score. This is useful when you only want to temporarily filter content. Otherwise the permanent scores will make up most of your configuration. To add a score, simply click Articles > Add a Scoring Rule…

Add Scoring Rule
Add a Scoring Rule

Here is an example of a rule you might add; this one greatly reduces the score of all entries that have potentially dangerous file extensions in the subject line:

Block Potentially Malicious Content
Block Potentially Malicious Content

Pan’s built in filter field allows you to sift through all of the articles you found with keywords. Pairing this functionality with the scoring one really shows off the power of Pan.

All created scores are kept in ~/.pan2/Scores so don’t worry if you mess one up. You can just as easily open this file and fix it. Any manual changes to this file will however require you to exit out of Pan (if it’s open) and restart it.

Here is just a few entries of what you might have in your Score file:

%BOS
% Greatly reduce score of potentially malicious content
[alt.bin*]
Score:: -9999
Subject: .*\.(exe|bat|vbs|cpl|msi|scr|vb(script)?|ws(f|h))[^A-Za-z0-9].*
%EOS

%BOS
% Moderately increase the score of compressed content
[alt.bin*]
Score:: 2500
Subject: .*\.(z(ip|[0-9]{2})|r(ar|[0-9]{2})|7z|iso)[^A-Za-z0-9]([0-9]{3}[^A-Za-z0-9])?.*
%EOS

%BOS
% Very slightly decrease the content of PAR content
% This allows it to not quite have the same spot light as
% the item it matches up against. If it were a compressed file
% it would already have +2500 from the previous score entry
% identified above.  These will just sit at +2400 instead.
[alt.bin*]
Score:: -100
Subject: .*(\.vol[0-9]+\+[0-9]+)?\.(par2|sfv)[^A-Za-z0-9].*
%EOS

%BOS
% Very slightly increase the score of NZB-Files
[alt.bin*]
Score:: 250
Subject: .*\.(nzb)[^A-Za-z0-9].*
%EOS 

%BOS
% Mildly drop the score of cross-posted content
[alt.bin*]
Score:: -750
Xref: (.*:){2} % cross-posted to 2 or more groups 
%EOS

Wrapping It Up

I’m certainly not asking anyone to change from their existing system if it works for them. What I am pointing out though is that Pan is completely free, it’s open source and the features it offers are comparable (if not better) than all of it’s competition. Although it works great on Linux, it also works on many other platforms as well such as Microsoft and Apple.

It might not have a beautiful interface, but it wasn’t built to fill your systems memory with bloated eye candy. It was built to be fast and effective… and truly, it really is.

The newer versions coming out are really great! If you haven’t given it a try since it’s dated ones, you really should! If you’re interested in seeing how Usenet is structured, than this is also a great tool to learn with. If you run an indexer (such as newznab or the many forks of it) you can practice your regular expressions (regexs) using Pan. For an Indexer Admin, this tool is especially great in debugging your regexs!

Credit

This blog took me a very long time to put together and test! The repository hosting alone accommodates all my blog entries up to this date. All of the custom packaging described here was done by me personally. I took the open source available to me and rebuilt it to make it an easier solution and decided to share it. If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

Sources