Tag Archives: FTP

Host Your Own WordPress Site

Introduction

A while back I decided to host my blog on my own servers and cut the cord with WordPress.com. There were no hard feelings really; I just didn’t like the limited options for plugin choices I had. I didn’t like all the extra features they forced me to use that made my blog (response time) slower.

The standalone version of WordPress is much more scaled down and fits my requirements better. The standalone version also allows me to pick and choose from a sea of amazing plugins at my disposal.

Prerequisite

You’ll need to have full (Linux) shell access to the server you’re intending to host this from. I set up my hosting using a CentOS 6.x machine; therefore the instructions I identify here are for that. That said, the instructions won’t stray too far off (requiring a tweak here an there) from those people who choose to use other Linux distributions.

You’ll want to first install a few packages:

# You'll basically need PHP, MySQL 
# As root (or a user with sudoer's permission) type the following:
sudo yum -y install php-fpm php-mbstring php-mcrypt \
             php-mysql nginx mysql mysql-server gawk \
             openssl

Optional FTP Support

FTP support is really nice to have with a WordPress setup! You can use it to automate the installation of plugins through the admin page. This is great for situations where you’re setting up a WordPress account for someone who isn’t too teksavvy. It also allows grants your users enough access to install plugins through WordPress’s administration interface.

I strongly suggest you read my blog entry on Configuring and Installing VSFTPD on CentOS 6 if you’re interested in going this route.

It’s important to note that WordPress functions just fine without an FTP(S) server too!

Step 1 of 8: Prepare our Environment.

To make life really easy (so you can cut and paste this right to your command line without any effort at all), lets create some environment variables.

Please note that this step MUST be ran before any of the other steps are. If you’re returning to this blog entry to resume from a step you left off at, be sure to apply these environment variables again!

Please note that you must be root or have sudoer’s permission to be able to perform any of these tasks successfully on your server.

# Our WordPress user
WPUSER=nuxref

# The FQD you will be serving your data from.  If you
# don't have your own domain, then set this to an
# underscore '_' (without the quotes '')
WPURL=nuxref.com

# Some Database Information
# - what are we going to call our database name?
DBNAME="wordpress_$WPUSER"
# - it's easier to just use the WordPress user account here
#   but if you want to change it to something else; here is
#   where you can do it:
DBUSER=$WPUSER
# - we will want to create a confusing password that others
#   can't guess. I don't recommend you use what i've identified
#   here because anyone else who knows you read my blog will
#   guess this first.  But here is where you should set your
#   database password you intend to use.
DBPASS="v3ryC0nFU51Ng-Pw%"

# If you plan on creating an FTP Account; you'll want to
# populate these variables too. This account does not have
# to be the same as the $WPUSER account. In fact making it
# different (even just slightly) would be a good idea!
# Below i just add '-ftp' to the end of the already
# determined user above.  Feel free to change this.
WPFTPUSER="$WPUSER-ftp"
# Set an FTP password; It would be a good idea to not use
# the one identified below as it's merely display only.
# some special character don't work with VSFTPD (like '!')
# if you plan on using it .
WPFTPPASS="4nt3rP455%rd"

# The following is only used for our SSL Key Generation
COUNTRY_CODE="7K"
PROV_STATE="Westerlands"
CITY="Lannisport"
SITE_NAME="Life as a Lannister"

Step 2 of 8: Create our User Account

You’ll want to create an isolated environment for our client (or you) to work within. By securing an environment; in the event anything is ever compromised, destruction will be limited to what we allow our client access to.

# First create a system directory to host our project.
sudo mkdir -p /opt/$WPUSER/html/static

# Create a dummy, favicon.ico file for now. If you feel
# ambitious, Google this if you're not sure what it's for
# so you can place your own custom one here
touch /opt/$WPUSER/html/static/favicon.ico

# Create System User
sudo useradd nuxref -M --system \
   --comment "$WPUSER WordPress Account" \
   --home /opt/$WPUSER \
   --shell /sbin/nologin

# Secure our new directory we created
chmod 711 /opt/$WPUSER
chown -R $WPUSER.$WPUSER /opt/$WPUSER

If you’ve followed my blog on Securing and Protecting Your CentOS 6 System then you might have wisely chosen to set up disk quotas. If not; then you can skip over to the next step.

# Detect the device using our home directory
DEV=$(df -l -P /opt/$WPUSER | awk 'END{print $1}')
# Restrict Users Disk Quota to 600MB
sudo setquota -u $WPUSER 180000 600000 0 0 $DEV

Step 3 of 8: Generate SSL Keys

We need to generate some Secure Socket Layer (SSL) keys so that we can provide a secure connection for logins. Otherwise our passwords we choose to work with the site could be exposed.

To make things simple, you can use my genssl tool first discussed in an earlier blog I wrote here. available for download from my github page and then just do the following:

# Generate a self signed key:
genssl -s $WPURL
# Install it:
sudo install -m 0400 $WPURL.key /etc/pki/tls/private/$WPUSER.key
sudo install -m 0444 $WPURL.crt /etc/pki/tls/certs/$WPUSER.crt

Or you can simply do the following:

# The following will generate SSL Keys (if you don't have any already)
sudo openssl req -nodes -new -x509 -days 730 -sha256 -newkey rsa:2048 
   -keyout /etc/pki/tls/private/$WPUSER.key 
   -out /etc/pki/tls/certs/$WPUSER.crt 
   -subj "/C=$COUNTRY_CODE/ST=$PROV_STATE/L=$CITY/O=$SITE_NAME/OU=IT/CN=$WPURL"
 
# Permissions; protect our Private Key
chmod 400 /etc/pki/tls/private/$WPUSER.key
 
# Permissions; protect our Public Key
chmod 444 /etc/pki/tls/certs/$WPUSER.crt

Step 4 of 8: Install our WordPress Bundle

Now we need to Download and install WordPress into our environment.

# WordPress Configuration
# Acquire latest version from here https://wordpress.org/download/
# (At the time it was 4.4.2)
wget --no-check-certificate https://wordpress.org/latest.tar.gz -O wordpress.$(date +'%Y.%m.%d').tgz

# Extract our downloaded copy
sudo tar xvfz wordpress.$(date +'%Y.%m.%d').tgz \
    -C /opt/$WPUSER/html --strip 1

# Apply some more permissions
sudo find /opt/$WPUSER/html -type d -exec chmod 755 {} \;
sudo find /opt/$WPUSER/html -type f -exec chmod 664 {} \;
sudo find /opt/$WPUSER/html -exec chown $WPUSER.apache {} \;

# Grant write permissions to a few tools our plugin installers
# will need access to later on:
find /opt/$WPUSER/html/wp-content/ -type d -exec chmod 775 {} \;
sudo chmod 660 /opt/$WPUSER/html/wp-config.php

Step 5 of 8: Configure and Prepare our Database

Now we need to configure our MySQL (or MariaDB) database. First make sure it is running:

# The below command will start the database if it isn't
# already running:
sudo service mysqld status &>/dev/null || \
   sudo service mysqld start

# Next make sure you're system is configured to start
# the database each and every time your server turns on
sudo chkconfig --level 345 mysqld on

Now we need to prepare our database that WordPress can use.

# SQL Initialization
( cat << _EOF
CREATE DATABASE $DBNAME;
GRANT ALL PRIVILEGES ON $DBNAME.* TO "$DBUSER"@"localhost" IDENTIFIED BY " $DBPASS";
FLUSH PRIVILEGES;
_EOF
) | sudo mysql

Step 6 of 8: Configure our Web Hosting Service

Okay now we need to host our website. Effectively linking the database we just prepared with the WordPress software we just installed. We do this as follows using NginX:

# By default (assuming a CentOS installation), we can
# plug into our configuration by writing our data in
# /etc/nginx/conf.d/
# So lets do just that:
cat << _EOF > /etc/nginx/conf.d/wordpress_$WPUSER.conf
#
# $WPUSER WordPress Web Hosting
#
server {
    # Support Web Traffic at port 80
    listen       80;
    server_name  $WPURL;
    root   /opt/$WPUSER/html;

    # Our log files
    access_log  /var/log/$WPUSER/$WPUSER.access.log  main;
    error_log  /var/log/$WPUSER/$WPUSER.error.log;

    # Our main handler
    location / {
        root   /opt/$WPUSER/html;
        index  index.html index.htm index.php;
        # Support Permalink changes
        try_files \$uri \$uri/ /index.php?q=\$request_uri;
    }

    # Anyone logging into our site should do it securely
    location /wp-admin/ {
       # Always redirect to secure site
       rewrite ^/(.*) https://$host/\$1 permanent;
    }
    location /wp-login/ {
       # Always redirect to secure site
       rewrite ^/(.*) https://\$host/\$1 permanent;
    }

    error_page  404              /404.html;
    location = /404.html {
        root   /usr/share/nginx/html;
    }

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # Support the favicon (for those wanting to use it)
    location = /favicon.ico {
        root   /opt/$WPUSER/html/static;
    }

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    location ~ \.php\$ {
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  \$document_root\$fastcgi_script_name;
        include        fastcgi_params;
    }

    # Deny access to the wp-config file
    location ~ /wp-config\.php {
        deny  all;
    }
}

server {
    # We should listen on a secure URL too so that we can
    # hide our admin login credentials from prying eyes
    listen       443;
    server_name  $WPURL;
    root   /opt/$WPUSER/html;

   ssl on;
   ssl_certificate /etc/pki/tls/certs/$WPUSER.crt;
   ssl_certificate_key /etc/pki/tls/private/$WPUSER.key;
   ssl_session_timeout  5m;

   # Secure our site by only allowing the TLS protocol
   ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
   ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128:AES256:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK';
   ssl_prefer_server_ciphers on;
   ssl_session_cache  builtin:1000  shared:SSL:10m;

   access_log  /var/log/nginx/$WPUSER.access.log  main;
   error_log  /var/log/nginx/$WPUSER.error.log;

   location / {
      root   /opt/$WPUSER/html;
      index  index.html index.htm index.php;
      # Support Permalink changes
      try_files \$uri \$uri/ /index.php?q=\$request_uri;
   }

   error_page  404              /404.html;
   location = /404.html {
      root   /usr/share/nginx/html;
   }

   # redirect server error pages to the static page /50x.html
   #
   error_page   500 502 503 504  /50x.html;
   location = /50x.html {
       root   /usr/share/nginx/html;
   }

   # Handle favicon
   location = /favicon.ico {
       root   /opt/$WPUSER/html/static;
   }

   # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
   #
   location ~ \.php\$ {
      fastcgi_pass   127.0.0.1:9000;
      fastcgi_index  index.php;
      fastcgi_param  SCRIPT_FILENAME  \$document_root\$fastcgi_script_name;
      include        fastcgi_params;
   }

   # Deny access to the wp-config file
   location ~ /wp-config\.php {
      deny  all;
   }
}
_EOF

Now restart our web services

# Ensure our web browser and php handler will start
# even if our server is restarted
chkconfig --levels 345 php-fpm on
chkconfig --levels 345 nginx on

# The following just makes sure we reload and take
# on our new configuration.  If we're not running
# then we start the services up
service php-fpm status &>/dev/null && \
   service php-fpm restart || \
   service php-fpm start

service nginx status &>/dev/null && \
   service nginx restart || \
   service nginx start

Step 7 of 8: Optionally Setup an FTP Account

Most people can skip this step; it again presumes you’ve followed my other blog on Configuring and Installing VSFTPD on CentOS 6. If you have not gone here or have set up FTP your own way, you can also skip this step and move on with Configuring WordPress.

# Create a WordPress Plugins FTP Account
echo $WPFTPUSER >> /etc/vsftpd/users.passwd
echo $WPFTPPASS >> /etc/vsftpd/users.passwd
 
# Protect Password
chmod 600 /etc/vsftpd/users.passwd
chown root.root /etc/vsftpd/users.passwd
 
# Now convert content into a db structure
db_load -T -t hash -f /etc/vsftpd/users.passwd /etc/vsftpd/virtual.users.db
chmod 600 /etc/vsftpd/virtual.users.db
chown root.root /etc/vsftpd/virtual.users.db
 
cat << _EOF > /etc/vsftpd/virtual.users/$WPFTPUSER
local_root=/opt/$WPUSER/html
# -------------------------------------------------------------------------
# User
# -------------------------------------------------------------------------
guest_enable=YES
guest_username=apache
local_root=/opt/$WPUSER/html
# -------------------------------------------------------------------------
# Permissions
# -------------------------------------------------------------------------
# write_enabled is required if the user is to make use of any of the
# anon_* commands below
write_enable=YES
# give the user the ability to make directories
anon_mkdir_write_enable=YES
# give the user the ability delete and overwrite files
anon_other_write_enable=YES
# give the user the ability upload new files
anon_upload_enable=YES
# Give the user permission to do a simple directory listings
dirlist_enable=YES
# Give the user permission to download files
download_enable=YES
# if the user has can upload or make new directories, then this will be
# the umask applied to them
anon_umask=0002
# delete failed uploads (speaks for itself)
delete_failed_uploads=NO
_EOF
 
sudo chmod 600 /etc/vsftpd/virtual.users/$WPFTPUSER
sudo chown root.root /etc/vsftpd/virtual.users/$WPFTPUSER

# Ensure our FTP Server will restart if our server
# is ever restarted:
sudo chkconfig --level 345 vsftpd on
# Update Service (to read in new configuration)
sudo service vsftpd status &>/dev/null && \
    service vsftpd restart || \
    service vsftpd start

Step 8 of 8: Configure WordPress

If you successfully pulled off all of the earlier steps, then you shouldn’t have any trouble from this point forward. The hard part is done with!

Wordpress Database SetupWordpress InstallationYou now need to open up your browser and access your new WordPress website to continue with the setup. Simply visit your website by browsing to http://your.wordpress.url/ (whatever you set this up as).

First you’ll be immediately presented with webpage that needs some information about the database we set up back in Step 5 (an environment variables defined in Step 1.

After you press the [Submit] button, you’ll then be asked to define some basic information about the blog you intend to set up. You can change all this later, so don’t worry. The important fields here are the administrator user and password you create.

You’re done now and ready to use WordPress

Great WordPress Plugins

The following plugins are worthy of a mention:

Credit

Please note that this information took me several days to put together and test thoroughly. I may not blog often; but I want to re-assure the stability and testing I put into everything I intend share.

If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least. It’s really all I ask.

UFTP: Mass File Distribution Using Multicasting

Introduction

Multicasting has it’s pros and cons just like everything else. But it is often an overlooked solution to a common business problem which is: How do I transfer a file to multiple (subscribed) locations at the same time?
Most administrators or developers will come up with a solution that involves sending this file to each site using a common protocol such as SFTP, SCP, RSYNC, FTP, RCP, etc. There is no doubt that these solutions will work. However these solutions require you to send the file to each location individually. Hence, if you need to send a 10MB file to 100 remote locations, you’ll need 1GB (10MB x 100) of local network bandwidth to do it with.
Traditional Protocol File Transfers
A Multicasting solution saves you this effort and bandwidth by allowing you to send the file once and have all 100 sites collectively store it onto their systems at (relatively) the same time. Now, with respect to the diagram below (and the rest of this blog), I’m focusing entirely on the amazing efforts Dennis Bush put into UFTP. It is this tool that makes all of this possible. UFTP is one of those diamonds in the rough that I don’t think gets enough attention for the value it brings with it.
Multicast File Transfer using UFTP

Multicast in a Nutshell

A Multicast boils down to just being an IP address anywhere in the 224.0.0.0/4 network range. It uses a User Datagram Protocol to communicate with anyone who chooses to read and write data from this address. Routers can be configured to share this address across networks so that everyone may join in. Similar to a chat room on the internet where everyone joins together and anything they write, everyone else in the channel can see too.

Multicasting no doubt has it’s drawbacks. But I figure that it’s not worth dwelling on what you can’t do with the (multicast) protocol on it’s own since the UFTP tool this blog focuses on for mass file distribution has solved many (if not all) of these problems for us.

Why Multicasting, Who Uses it?

Multicasting saves bandwidth. Think of your Cable TV provider; when you change a channel, you’re actually just subscribing to a new multicast address along with the 100+ million other subscribing customers. Multicasting allows Cable TV companies to broadcast every channel at once, and those who are interested in a specific channel will receive it. Regardless of what channel you change to, there is no additional load sustained by the cable provider.

System administrators may not even know they’re using it when if they are managing a cluster. Most clusters use multicast as a way of passing their heartbeats to all other nodes in efforts to keep quorum.

Now, with respect to UFTP, if you visit it’s official website, you’ll see that the the Wall Street Journal used this tool in the past to send their developed newspapers to remote printing plants and other outlets across the United States.

UFTP in a Nutshell

UFTP is a File Transfer solution that wraps itself around the the multicast protocol as well as addressing the deficiencies that come with it. It is a well designed server/client application that allows for one to easily transfer files from 1 location to as many clients as you want in one shot. It can drastically save your company on bandwidth and has been around long enough to be deemed a reliable business decision. It’s worth noting that UFTP was written in C making it incredibly fast and lightweight on system resources. It works on all platforms but I’ll specifically focus on CentOS and the rpms I’ve packaged.

To use the software, you simply pass it a file and it looks after all the dirty work of guaranteeing it’s delivery to all of the clients subscribed to the address it broadcasts on. The author thought of everything when developing his tool, encryption of the data is always available to you as well if you prefer. He developed 3 main tools that you can manipulate to feed your data anywhere.
A Simple UFTP Environment

  • uftp (The Server – Docs): A simple command line tool that takes a file and broadcasts it to all of the listening clients. This is the exact reverse of the traditional ftp, sftp, scp, etc tools where they become the client and need to connect to a server to preform their tasks.
  • uftpd (The Client – Docs): A daemon that starts up and listens to a multicast address for new files being broadcasted by the server. Again, you’ll notice with Multicasting, roles are reversed from what you’d normally be used to. With the traditional protocols (FTP, SFTP, SCP, etc), they usually have daemons to host the server side of things not the client. You’ll want to run this daemon at all locations you wish to receive content broadcasted by the server (uftp).
  • uftpproxyd (The Proxy – Docs): The proxy allows you to tunnel your uftp multicast across a network that doesn’t support multicasting (such as the internet). This allows clients in other controlled networks (separated by one you can’t control) to additionally be part of your file distribution.

    The UFTP Proxy has 3 main modes it operates as:

    • Server Mode: This is used for pushing content upstream. This would effectively sit on the same server you call ‘uftp’ from. It listens just like any other client would and passes all information it receives to connected UFTP Proxies configured with the client mode.
    • Client Mode: This communicates with a UFTP Proxy configured for Server Mode and mimics the ‘uftp’ by broadcasting the same data to the local multicast address. This allows all local UFTP Clients to retrieve the file(s).
    • Response Mode: This is used to help take off some of the load of the server if there are many clients. Although the file is only being broadcasted once, there is a lot of handshaking that goes on at the start and end of the transmission to guarantee all data was delivered successfully. Depending on the different networks, their medium and reliability, a server may need some extra help with the handshaking if there are a lot of clients involved struggling to retrieve the data.

    Below shows an example of how the proxy can be utilized:
    A UFTP Proxy ConfigurationNote: Pinholes are used as a way to connect back to the UFTP Proxy Server effectively requiring no firewall changes to be made at each client site (only the server).

    **Note: It’s important to note a single Proxy Server can only be configured to connect to a single Proxy Client; it’s a 1 to 1 mapping. If you have multiple sites you need to connect to, you’ll need to set up an individual Proxy Server for each Proxy Client you need to serve.

    If you’re interested in the proxy portion of UFTP, you can read the official documentation about it found here.

Hand Over Everything

I wouldn’t have it any other way:

  • uftp-4.5.1-1.el6.nuxref.x86_64.rpm: The server is just the uftp tool and is really easy to use. This assumes you’ve got clients configured somewhere listening though!
    # just type uftp file.you.want.to.send
    uftp mytestfile
    
    # You can also send multiple files by just adding them to the end of
    # the string:
    uftp mytestfile1 mytestfile2 mytestfile3
    
    # I also wrote a small script that works the same way and sends stuff encrypted
    uftpe mytestfile1 mytestfile2 mytestfile3
    
  • uftp-client-4.5.1-1.el6.nuxref.x86_64.rpm: The client listens for data sent from the uftp server. You can use the RC Script i prepared to greatly simplify this tool:
    # as root; use the RC Script I wrote to make hosting the server
    # really easy
    service uftpd start
    

    Filtering is optional; if you don’t specify any, then by default there are no restrictions. Sometimes this is satisfactory (especially in closed or isolated networks). I attempted to simply UFTP’s built in server filtering for those who want to use it though. You see, not only can you encrypt the data you transmit from the server. But you can restrict the client to only accept connections from specific UFTP servers residing at a specific hosts with a specific server id. You can even go as far as only accepting servers with a specific fingerprint (created by their private key)

    # simply drop a configuration file in /etc/uftpd/servers.d 
    # Examples of accepted entries (taken from uftpd man page):
    # 0x11112222|192.168.1.101|66:1E:C9:1D:FC:99:DB:60:B0:1A:F0:8F:CA:F4:28:27:A6:BE:94:BC
    # 0x11113333|fe80::213:72ff:fed6:69ca
    #
    # You can have as many files as you want in this directory with as many entries in each
    # as you want.  If you add or remove new files, you'll need to restart the uftpd service
    # since it's only read in at the start.
    

    So by adding this bit of complexity, I know you are asking yourself:

    • Q: How do I know what my Server ID is?
      A: By default (using the rpm I’ve packaged) every server has an ID of 0x00000001 (decimal value of 1 – one) if you use the uftpe script I wrote (for encrypted transfers). To change your server id do the following:

      # Define a new id (other then 1)
      NEWID=2
      
      # Simply store this ID inside of the $HOME/.uftp config
      # file as it's HEX value:
      
      # Ensure the config directory exists
      [ ! -d $HOME/.uftp ] && mkdir -p  $HOME/.uftp
      
      # clear any old entry you may have set if you want:
      [ -f $HOME/.uftp/config ] && sed -i -e '/^UFTP_UID=/d' $HOME/.uftp/config
      
      # Set the new entry
      printf 'UFTP_UID=0x%.8xn' $NEWID >> $HOME/.uftp/config
      

      If however your just using the uftp tool on it’s own, it takes on the IP address of the host it’s running on (as it’s hex value) unless you explicitly specify -U 0x00000002 (or whatever ID you want it to assume). Here is a quick example of how you can convert an IP address to it’s hex value at the shell:

      # Define the address you want to convert
      IP_ADDR=192.168.1.128
      # Now convert it (don't forget the brackets!)
      (
         printf '0x'
         printf '%02X' $(echo "${IP_ADDR//./ }"); echo
      )
      # example above outputs: 0xC0A80180
      
    • Q: How do I know what my Server Fingerprint is?
      A: First off, uftp will not ever connect to this server with this option set if you choose ‘not’ to use the uftpe script i wrote or provide the necessary switches to enforce encryption. Setting a filter of your servers fingerprint is also a way of saying you only accept connections that are encrypted. This entry is completely optional. Your fingerprint ID is stored in $HOME/.uftp/uftp.key. Again this key only exists if your using the uftpe tool; otherwise the key is wherever you chose to store it. Fetch the id as follows (the below is intended to be called on the server where the uftp.key file exists):

      # fetch the details from the key:
      uftp_keymgt $HOME/.uftp/uftp.key
      

      Don’t panic if you don’t have a key; They are generated automatically when you first run the uftpe tool. The easiest way to pre-create it would just be to call uftpe by itself (without any parameters). Yes; it’ll spit an error telling you you didn’t provide it enough options. But the script will also generate you a key automatically too.

    I tried to think of everything; so log rotations and logging is already built in and included. You can locate them in /var/log/uftpd.log.

  • uftp-proxy-4.5.1-1.el6.nuxref.x86_64.rpm: the proxy service can bridge two networks that don’t support multicasting together. I haven’t spent to much time with this area since my environment hasn’t required me to. If you have any information to share about it; feel free and I can expand this area.
  • uftp-debuginfo-4.5.1-1.el6.nuxref.x86_64.rpm (Optional): This is only required if you are debugging this tool.
  • uftp-4.5.1-1.el6.nuxref.src.rpm (Optional): The source RPM for those interested in building the software for themselves.

Some Things To Consider:

When something is explained to be this easy, there is always a catch. I won’t lie, there are a few which means the UFTP solution may not be for everyone. They are as follows:

  • Multicasting isn’t enabled by most routers by default. If your recipients reside in a network you don’t manage, you’ll want to ask the local administrator to make sure their routers have (Level 2) multicasting enabled.
  • Multicasting can be a pain in the butt to troubleshoot; although newer routers won’t give you any grief, some of the older ones can fail to relay content correctly to clients who’ve also connected to the same multicast address. With negativity aside though, when it works, it works so great! My point is: you’ll need to make sure you’re using (networking) hardware that is relatively new (no older then 2010) where the firmware adequately supports Level 2 multicasting.
  • The UFTP Proxy attempts to resolve the problem of linking your networks together when separated by ones you don’t control. But consider that unless there are multiple recipients located on each network you connect your proxy too, you aren’t saving any bandwidth choosing this route.
  • Only uftp clients who are online will receive content sent by the uftp server. This can be a deal breaker for some especially if the product being delivered ‘MUST’ reach all of the clients. UFTP does not track who is online and who isn’t. It simply delivers content to those who are present at that time. It’s just like how you can’t watch a television show if you haven’t told your TV’s receiver what channel to be on.
  • If you’re using restrictive firewall settings (hopefully you are!), you’ll want to make sure multicasting is allowed into your client box with the following:
    iptables -A INPUT -m pkttype --pkt-type multicast -j ACCEPT

    It would be worth adding this to your /etc/sysconfig/iptables file as well.

Repository

Please note that all of these packages are also within my repository if that makes things easier for you and your deployment.

Sources

  • UFTP: Dennis Bush did a fantastic job with this tool. It is the key to making multicast file transmission powerful and reliable.
  • Multicasting: information in greater depth can be found here.

Configuring and Installing VSFTPD on CentOS 6

Introduction

FTP Servers have been around for a very long time. On could easily argue that they aren’t always the best option to choose anymore. But for those dealing with legacy systems, the FTP protocol provides one the easiest ways to share and distribute files across external systems. Despite the hate some might give the protocol; it still receives a lot a love from others just because it’s still very compatible with just about everything; heck, even web browsers such as Internet Explorer, Chrome, and Firefox (and many more) have the protocol built right into it.

In my case, I needed to set up an FTP server to help a client with some legacy software they use (and are familiar with). This blog is more or less just the steps I took to make it work in case anyone else is interested.

Security Concerns

FTP preforms all of it’s transactions in ‘plain text’ including it’s authentication. This wasn’t a problem back in 1980 when online security wasn’t an issue. This also isn’t a problem for sites offering anonymous file hosting services. But for everyone else, it pretty much leaves you susceptible to privacy issues and possible intrusions.

FTPS (not to be confused with SFTP) is a way of securing the FTP protocol for the systems that require it allowing you to eliminate the ‘plain text’ problem. But this requires the client uses software that can take advantage of it.

Some additional security concerns I wanted to consider:

  • Separate user accounts from the system ones. We don’t want people trying to guess our root password or access anyone’s home directory unless with specifically configure the server to allow it.

Setup VSFTPD

VSFTPD stands for Very Secure FTP Daemon and provides all the flexibility we need. It’s official website can be found here.

The following will set up VSFTPD into your CentOS/RedHat environment in a isolated manor. You see we want to disconnect the users that currently access your system from the users we create for VSFTPD for more flexibility and control.

# First fetch the required packages
# db4:       The Berkeley Database (Berkeley DB) which is a
#             quick and dirty way of storing our user accounts.
# db4-utils: This provides a tool we'll use to build a small user
#             database with.  Technically you can uninstall (just)
#             this package afterwards for added security after.
# openssl:   This is used to grant our FTP server FTPS support
# vsftpd:    The FTP/FTPS Daemon itself
yum -y install db4 db4-utils openssl vsftpd

# Create a password file: /etc/vsftpd/users.passwd
# This file contains all of the users you want to allow on the
# site in the structure:
# Line: Entry
#  1  | USERNAME1
#  2  | PASSWORD1
#  3  | USERNAME2
#  4  | PASSWORD2
# This below creates a simple user 'foobar' and a password
# of 'barfoo'.  This isn't the safest way to build your password
# file because it leaves a persons password set available in
# the shell history... But for the purpose of this tutorial:
echo foobar > /etc/vsftpd/users.passwd
echo barfoo >> /etc/vsftpd/users.passwd

# Protect our password file now from prying eyes:
chmod 600 /etc/vsftpd/users.passwd
chown root.root /etc/vsftpd/users.passwd

# Prepare a directory we want the foobar user to send it's files
# to. You can also just use a directory you already have in place
# or someone elses home directory.
mkdir -p /var/ftp/foobar

# Set a comfortable permission to this directory granting the ftp
# user (or any user account your later going to assign to this user
# read/write access)
chown root.ftp /var/ftp/foobar
chmod 775 /var/ftp/foobar

# Convert our plain-text password file into the Berkeley Database
# format. This is the only command that requires the db4-utils
# rpm package which you can uninstall if you want for security
# reasons after you run the below command.  You'll need to
# re-install it though if you ever want to add or update accounts
db_load -T -t hash 
   -f /etc/vsftpd/users.passwd /etc/vsftpd/virtual.users.db

# Protect our new (Berkley) database:
chmod 600 /etc/vsftpd/virtual.users.db
chown root.root /etc/vsftpd/virtual.users.db

# Prepare our virtual user directory; this is where we can
# optionally place over-riding configuration for each user
# we create in the Berkley database above.
mkdir -p /etc/vsftpd/virtual.users
chmod 700 /etc/vsftpd/virtual.users
chown root.root /etc/vsftpd/virtual.users

# Create PAM Module that points to our new database.
# Note: you do not provide the '.db' extension when creating
#       this file. The file is valid as you see it below.
cat << _EOF > /etc/pam.d/vsftpd-virtual
auth     required pam_userdb.so db=/etc/vsftpd/virtual.users
account  required pam_userdb.so db=/etc/vsftpd/virtual.users
session  required pam_loginuid.so
_EOF

# Protect our Module
chmod 644 /etc/pam.d/vsftpd-virtual
chown root.root /etc/pam.d/vsftpd-virtual

# Create an empty jail directory.  This is used for default
# configurations only. A well configured system won't even use
# this; but it's still good to have since we'll be referencing
# it in our configuration. This will become the default
# directory a user connects to if they aren't otherwise
# configured to go to another location.
mkdir -p /var/empty/vsftpd/
chown nobody.ftp /var/empty/vsftpd/
chmod 555 /var/empty/vsftpd/

# Now we want to allow FTPS support, we'll need an SSL key to
# do it with.  If you already have one, you can skip this step.
# Otherwise, the following will just generate you a self-signed
# key as a temporary solution.
openssl req -nodes -new -x509 -days 730 -sha256 -newkey rsa:2048 
   -keyout /etc/pki/tls/private/nuxref.com.key 
   -out /etc/pki/tls/certs/nuxref.com.crt 
   -subj "/C=7K/ST=Westerlands/L=Lannisport/O=NuxRef/OU=IT/CN=nuxref.com"

# Protect our Keys
chmod 400 /etc/pki/tls/private/nuxref.com.key; # Private Key
chmod 444 /etc/pki/tls/certs/nuxref.com.crt; # Public Certificate

# Create ourselves a little banner we can use to at least alert
# human intruders that they are in fact being monitored.  This
# scare tactic may or may not work, but if you ever have a breach
# of security, you may need to reference that you gave the user
# ample warning that they were violating someones rights by
# continuing.  Feel free to adjust the banner to your liking.
cat << _EOF > /etc/banner
* - - - - - - W A R N I N G - - - - - - - W A R N I N G - - - - - *
*                                                                 *
* The use of this system is restricted to authorized users. All   *
* information and communications on this system are subject to    *
* review, monitoring and recording at any time, without notice or *
* permission.                                                     *
*                                                                 *
* Unauthorized access or use shall be subject to prosecution.     *
*                                                                 *
* - - - - - - W A R N I N G - - - - - - - W A R N I N G - - - - - *
_EOF

# Protect our banner
chmod 640 /etc/banner

At this point we have our environment set up the way we want it. The next step is to create our VSFTPD configuration.

# Lets first backup original configuration file
mv /etc/vsftpd/vsftpd.conf /etc/vsftpd/vsftpd.conf.orig

# Create new configuration
cat << _EOF > /etc/vsftpd/vsftpd.conf
# --------------------------------------------------------------
# Base Configuration
# --------------------------------------------------------------
anon_world_readable_only=NO
anonymous_enable=NO
chroot_local_user=YES
hide_ids=YES
listen=YES
local_enable=YES
max_clients=10
max_per_ip=3
nopriv_user=ftp
pasv_min_port=64000
pasv_max_port=64100
session_support=NO
user_config_dir=/etc/vsftpd/virtual.users
userlist_enable=YES
use_localtime=YES
xferlog_enable=YES
xferlog_std_format=NO
log_ftp_protocol=YES
pam_service_name=vsftpd-virtual
banner_file=/etc/banner
reverse_lookup_enable=NO
# --------------------------------------------------------------
# Secure Configuration (FTPS)
# --------------------------------------------------------------
ssl_enable=YES
virtual_use_local_privs=NO
allow_anon_ssl=NO
# forcing SSL makes the FTP portion of your site disabled and it
# will only operate as FTPS.  This may or may not be what you
# want.
force_local_data_ssl=NO
force_local_logins_ssl=NO
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
# Point to our certificates
rsa_cert_file=/etc/pki/tls/certs/nuxref.com.crt
rsa_private_key_file=/etc/pki/tls/private/nuxref.com.key
require_ssl_reuse=NO
ssl_ciphers=HIGH:!MD5:!ADH
# --------------------------------------------------------------
# FTP Configuration
# --------------------------------------------------------------
async_abor_enable=YES
ftp_data_port=20
connect_from_port_20=YES
# --------------------------------------------------------------
# Default Anonymous Restrictions (over-ride per virtual user)
# --------------------------------------------------------------
guest_enable=NO
guest_username=nobody
# Default home directory once logged in
local_root=/var/empty/vsftpd
# write_enabled is required if the user is to make use of any of
# the anon_* commands below
write_enable=NO
# give the user the ability to make directories
anon_mkdir_write_enable=NO
# give the user the ability delete and overwrite files
anon_other_write_enable=NO
# give the user the ability upload new files
anon_upload_enable=NO
# Give the user permission to do a simple directory listings
dirlist_enable=NO
# Give the user permission to download files
download_enable=NO
# if the user has can upload or make new directories, then this
# will be the umask applied to them
anon_umask=0002
# delete failed uploads (speaks for itself)
delete_failed_uploads=YES
_EOF

# Protect our configuration
chmod 600 /etc/vsftpd/vsftpd.conf
chown root.root /etc/vsftpd/vsftpd.conf

Technically we’re done now, but because we intentionally specified very restrictive user access rights, our foobar user we created will only connect to the /var/empty/vsftpd directory with no access rights. Therefore, our final step is to create an additional configuration file for the foobar account granting him read/write access to /var/ftp/foobar.

# The file you write to in the /etc/vsftpd/virtual.users/ 'must' be the same
# name as the user(s) you created to over-ride their permissions!
cat << _EOF > /etc/vsftpd/virtual.users/foobar
local_root=/var/ftp/foobar
# --------------------------------------------------------------
# User
# --------------------------------------------------------------
guest_enable=YES
# Set this to any system user you want
guest_username=ftp
local_root=/var/ftp/foobar
# --------------------------------------------------------------
# Permissions
# --------------------------------------------------------------
# write_enabled is required if the user is to make use of any of
# the anon_* commands below
write_enable=YES
# give the user the ability to make directories
anon_mkdir_write_enable=YES
# give the user the ability delete and overwrite files
anon_other_write_enable=YES
# give the user the ability upload new files
anon_upload_enable=YES
# Give the user permission to do a simple directory listings
dirlist_enable=YES
# Give the user permission to download files
download_enable=YES
# if the user has can upload or make new directories, then this
# will be the umask applied to them
anon_umask=0002
# delete failed uploads (speaks for itself)
delete_failed_uploads=NO
_EOF

# Protect our foobar permission file
chmod 600 /etc/vsftpd/virtual.users/foobar
chown root.root /etc/vsftpd/virtual.users/foobar

You are now complete, you can start VSFTPD at any time:

# Have VSFTPD start after each system reboot
chkconfig --level 345 vsftpd on

# Start VSFTPD if it isn't already running
service vsftpd status || service vsftpd start

It’s worth noting that if you ever change any of the configuration or add more, you will need to restart the VSFTPD server in order for the changes you made to take effect:

# Restarting the vsftpd server is simple:
service vsftpd restart

Firewall Configuration

The FTP firewall configuration can get complicated especially when the ephemeral ports it chooses to open are random (when operating it in Passive mode). If you scan through the configuration file above, you’ll see that we’ve specified this range to be between 64000 and 64100.

In a nutshell, if you choose to use/enable FTPS (which I strongly recommend you do), the firewall configuration will look like this (/etc/sysconfig/iptables):

#....
#---------------------------------------------------------------
# FTP Traffic
#---------------------------------------------------------------
-A INPUT -p tcp -m state --state NEW --dport 21 -j ACCEPT
# non-encrypted ftp connections (ip_contrack_ftp) module looks
# after these ports, however for the encrypted sessions it can't
# spy so we need to disable ip_contrack_ftp and just open the
# port range himself
-A INPUT -p tcp -m state --state NEW --dport 64000:64100 -j ACCEPT
#...

However, if (and only if) you choose not to use FTPS and strictly operate using FTP only then your configuration will look as follows:

  1. /etc/sysconfig/iptables
    #....
    #---------------------------------------------------------------
    # FTP Traffic
    #---------------------------------------------------------------
    -A INPUT -p tcp -m state --state NEW --dport 21 -j ACCEPT
    #...
  2. /etc/sysconfig/iptables-config
    This file requires you to add ip_conntrack_ftp to the variable IPTABLES_MODULES which is near the top of this file. You may also need to update /etc/sysconfig/ip6tables-config if you are using ip6; the change is the same. This keeps the entire range of 64000 to 64100 ports closed be default and through packet sniffing, they are opened on demand.
#...
IPTABLES_MODULES="ip_conntrack_ftp"
#...

If you’re not running any other modules, you can use the following one liner to update the file:

sed -i -e 's/^IPTABLES_MODULES=.*/IPTABLES_MODULES="ip_conntrack_ftp"/g' 
    /etc/sysconfig/iptables-config
sed -i -e 's/^IPTABLES_MODULES=.*/IPTABLES_MODULES="ip_conntrack_ftp"/g' 
    /etc/sysconfig/ip6tables-config

Be sure to reload your firewall configuration once you have these in place:

# Restart the firewall
service iptables restart

Fail2Ban Brute Force Configuration

The final thing you should consider if your server will be available via the internet is some bruit force prevention. I really recommend you read my blog on Securing Your CentOS 6 System, specifically my blurb on Fail2Ban which I think all systems should always have running. Fail2ban allows you to track all users hitting your FTP server and take immediate action on preventing further access to this potential intruder.

The configuration is as follows (remember to set the email to what you want it as where I’ve specified your@email.goes.here so you can be notified of any intrusions that take place.

cat << _EOF >> /etc/fail2ban/jail.conf
[my-vsftpd-iptables]

enabled  = true
filter   = vsftpd
action   = iptables[name=VSFTPD, port=ftp, protocol=tcp]
           sendmail-whois[name=VSFTPD, dest=your@email.goes.here]
logpath  = /var/log/vsftpd.log
_EOF

Be sure to reload your Fail2Ban configuration once this is done

# Restart Fail2Ban
service fail2ban restart

Test Our Configuration

Make sure you have an ftp tool installed into your environment like lftp or even a GUI based tool like FileZilla that supports both FTP and FTPS. The old Linux tool ‘ftp’ will only allow you to test the un-encrypted connection.

# Install lftp to keep things simple
yum -y install lftp

# Test our server (FTP)
[root@nuxref ~]# lftp ftp://foobar:barfoo@localhost
lftp foobar@localhost:~> pwd
ftp://foobar:barfoo@localhost
lftp foobar@localhost:~> exit

# Test our server (FTPS)
[root@nuxref ~]# lftp ftps://foobar:barfoo@localhost
lftp foobar@localhost:~> pwd
ftps://foobar:barfoo@localhost
lftp foobar@localhost:~> exit

First lets just test the basic FTP portion (plain-text):

# Connect to our server
[root@nuxref ~]# ftp localhost
Connected to localhost (127.0.0.1).
220-* - - - - - - W A R N I N G - - - - - - - W A R N I N G - - - - - *
220-*                                                                 *
220-* The use of this system is restricted to authorized users. All   *
220-* information and communications on this system are subject to    *
220-* review, monitoring and recording at any time, without notice or *
220-* permission.                                                     *
220-*                                                                 *
220-* Unauthorized access or use shall be subject to prosecution.     *
220-*                                                                 *
220-* - - - - - - W A R N I N G - - - - - - - W A R N I N G - - - - - *
220 
Name (localhost:nuxref): foobar
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
227 Entering Passive Mode (127,0,0,1,250,83).
150 Here comes the directory listing.
226 Directory send OK.
FileZilla FTPS Configuration
FileZilla FTPS Configuration

If you plan on using FileZilla as your solution, You need to configure it to connect as the FTP protocol with the Encryption set to Requires explicit FTP over TLS similar to the screen shot I provided.

You may or may not have to accept your certificate afterwards that we created earlier in this blog.

FileZilla On-Going FTPS Bug

The FileZilla Client is a pretty sweet application for those who like to work with a GUI instead of a command line. Those who choose to test their configuration with this should just know that there is an outstanding bug with FileZilla and the FTPS protocol. Hence, if you’re using Filezilla to to test your new VSFTPD server and it’s not working, it might not be your configuration at the end of the day. The versions seem to be hit and miss of which cause the bug to surface; reports of v3.5.2 working and v3.5.3 not. That all said, I’m using v3.7.3 and am not having a problem.

Here is the ticket #7873 that identifies the problem. One thing that is mentioned is that an earlier version of FileZilla works perfectly fine (specifically v3.5.2). But I’ve also had no problem with the current version (at the time was v3.7.3). I guess my main point is… don’t panic if you see this error; it’s not necessarily anything you’ve configured incorrectly. If you followed this blog then you shouldn’t have any issue at all.

Credit

I may not blog often; but I want to re-assure the stability and testing I put into everything I intend share.

If you like what you see and wish to copy and paste this HOWTO, please reference back to this blog post at the very least; it’s really all that I ask of you.

Sources: