Monit is a utility that you can set up to monitor your server’s processes and files, alert you to problems, restart stopped services, and present a web interface to display the server’s current status. I suspect most administrators would configure it to email them when services are down, drives are nearly full, load is too high, and so on. I don’t bother with email alerts, but I really like Monit’s web front-end and service restarting capabilities.

Monit system monitoring (yes, my server is redlining!)

Install Monit

Installing Monit on an Ubuntu server is accomplished in the usual way.

$ sudo apt-get install monit

Configure Monit

You have to edit a configuration file to start monit on your server.

$ sudo nano /etc/defaults/monit

Default configuration file

# Defaults for monit initscript
# sourced by /etc/init.d/monit
# installed at /etc/default/monit by maintainer scripts
# Stefan Alfredsson

# You must set this variable to for monit to start
startup=0

# To change the intervals which monit should run,
# edit the configuration file /etc/monit/monitrc
# It can no longer be configured here.

You have to change “startup=0″ to “startup=1″.

Edit the Control File

Monit’s control file is the place to set up system, service, and file monitoring. The default control file is all commented out. When I set mine up, I uncommented various lines and inserted sections for each service at the bottom of the file.

$ sudo nano /etc/monit/monitrc

Edit the file (uncomment lines), to enable the following options, which set up a monitoring daemon and enables the web interface (http://localhost:2812).

set daemon  120

set logfile syslog facility log_daemon

set httpd port 2812
allow localhost
allow 192.168.1.0/255.255.255.0

check system localhost

if loadavg (1min) > 4 then alert
if loadavg (5min) > 2 then alert
if memory usage > 75% then alert
if cpu usage (user) > 70% then alert
if cpu usage (system) > 30% then alert
if cpu usage (wait) > 20% then alert

check filesystem root with path /

Add the following sections to the bottom of the file to monitor cron, sshd, samba, mediatomb, lighttpd, and MySQL (which I am running to sync XMBC across multiple PCs). Services that run with a pid file are simple to monitor. MySQL does not run with a pidfile, but can easily be configured to by adding the line "pid-file = /var/run/mysqld/mysqld.pid" to the [mysqlid] section of /etc/mysql/my.cnf.

# cron
check process cron with pidfile /var/run/crond.pid
group system
start program = "/etc/init.d/cron start"
stop  program = "/etc/init.d/cron stop"
if 5 restarts within 5 cycles then timeout
depends on cron_rc

check file cron_rc with path /etc/init.d/cron
group system
if failed checksum then unmonitor
if failed permission 755 then unmonitor
if failed uid root then unmonitor
if failed gid root then unmonitor

# sshd
check process sshd with pidfile /var/run/sshd.pid
start program  "/etc/init.d/ssh start"
stop program  "/etc/init.d/ssh stop"
if failed port 22 protocol ssh then restart
if 5 restarts within 5 cycles then timeout

#  samba
check process smbd with pidfile /var/run/samba/smbd.pid
group samba
start program = "/etc/init.d/smbd start"
stop  program = "/etc/init.d/smbd stop"
if failed host 192.168.1.3 port 139 type TCP  then restart
if 5 restarts within 5 cycles then timeout

# mediatomb
check process mediatomb with pidfile /var/run/mediatomb.pid
group mediatomb
start program = "/etc/init.d/mediatomb start"
stop  program = "/etc/init.d/mediatomb stop"
if 5 restarts within 5 cycles then timeout

# lighttpd
check process lighttpd with pidfile /var/run/lighttpd.pid
group lighttpd
start program = "/etc/init.d/lighttpd start"
stop program = "/etc/init.d/lighttpd stop"
if failed host 127.0.0.1 port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout

# mysql
# note: requires mysql to run with a pid.
check process mysql with pidfile /var/run/mysqld/mysqld.pid
group database
start program = "/etc/init.d/mysql start"
stop program = "/etc/init.d/mysql stop"
if failed host 127.0.0.1 port 3306 then restart
if 5 restarts within 5 cycles then timeout

Unfortunately, monit can’t monitor the transmission-daemon service, because transmission-daemon does not run with a pid file, and forks itself sometimes. There is a warning in the transmission-daemon launch script, /etc/init.d/transmission-daemon, which says that pid file support will not work, because the daemon forks. Hopefully this will be resolved in the future.

Using Monit

After editing your configuration file, restart monit.

$ sudo service monit restart

Now you can check the health of your server by pointing a web browser to your server’s IP address, port 2812 (http://server:2812/).

Left alone, these completed torrents will not be removed from the server, and will accumulate as you download more torrents over time. There is no automated way to remove these torrents. Also, there is no automated way to move files to a new folder after seeding is complete, only after downloading is complete.

My ultimate goals are to complete seeding up until my seeding ratio is hit, to move torrents out of the seeding folder once seeding is complete, and to use the TVNamer Python script to automatically move any TV episodes I have downloaded to my media server’s TV folder.

Because I want to use TVNamer to move and rename most of my downloads automatically, but I want them to be seeded up to the ratio I set, I cannot use TVNamer on my “seeding” folder. If I did so, there’s a big chance that files would be removed from my “seeding” folder before seeding was complete. To solve that, I decided to move all completed torrent downloads to another folder, “complete”, and set TVNamer to run on that folder. To move completed torrents, I created a Transmission queue management script, which performs the following actions every 15 minutes:

1. Check for completed downloads that have either been paused manually or have seeded until the seeding ratio.
2. Move the downloaded files to another folder (“complete”).
3. Remove the torrent from the Transmission server.

Removing Completed Torrents Using a Script

Here is the script, which is based on my previous version. Compared to my last script, this one’s largest difference is the elimination of the netrc argument in the transmission-remote calls. I noticed that I did not need to send credentials to the Transmission daemon, presumably because my user account has all the privileges it needs to run transmission-remote.

A commenter on my previous post of this script informed me that Transmission 2.x uses “Finished” instead of “Stopped” to indicate completed torrents. I incorporated that change into the script, so that it will work with the old or the new verbiage.

Be sure to change the value of the MOVEDIR to the path you wish to move the downloads to. If you set it to “%1”, you can specify the path as a command-line argument.

Script file: /home/mjdescy/bin/removecompletedtorrents

#!/bin/sh

# script to check for complete torrents in transmission folder, then stop and move them

# either hard-code the MOVEDIR variable here…
MOVEDIR=/home/mjdescy/media # the folder to move completed downloads to
# …or set MOVEDIR using the first command-line argument
# MOVEDIR=%1

# use transmission-remote to get torrent list from transmission-remote list
# use sed to delete first / last line of output, and remove leading spaces
# use cut to get first field from each line
TORRENTLIST=`transmission-remote –list | sed -e ’1d;$d;s/^ *//’ | cut –only-delimited –delimiter= ” ” –fields=1`

# for each torrent in the list
for TORRENTID in $TORRENTLIST
do
echo “* * * * * Operations on torrent ID $TORRENTID starting. * * * * *”

# check if torrent download is completed
DL_COMPLETED=`transmission-remote –torrent $TORRENTID –info | grep “Percent Done: 100%”`

# check torrent’s current state is “Stopped”, “Finished”, or “Idle”
STATE_STOPPED=`transmission-remote –torrent $TORRENTID –info | grep “State: Stopped\|Finished\|Idle”`

# if the torrent is “Stopped”, “Finished”, or “Idle” after downloading 100%…
if [ "$DL_COMPLETED" != "" ] && [ "$STATE_STOPPED" != "" ]; then
# move the files and remove the torrent from Transmission
echo “Torrent #$TORRENTID is completed.”
echo “Moving downloaded file(s) to $MOVEDIR.”
transmission-remote –torrent $TORRENTID –move $MOVEDIR
echo “Removing torrent from list.”
transmission-remote –torrent $TORRENTID –remove
else
echo “Torrent #$TORRENTID is not completed. Ignoring.”
fi

echo “* * * * * Operations on torrent ID $TORRENTID completed. * * * * *”

done

After creating the script file, be sure to make it executable.

$ sudo chmod u+x /home/mjdescy/bin/removecompletedtorrents

Scheduling the Script

Schedule execution of the script using cron.

$ crontab -e

I run the script every 10 minutes, with a 1-minute offset from the top of the hour. I redirect output to /dev/null so cron will not send me emails regarding the script’s output.

1,11,21,31,41,51 * * * * /home/mjdescy/bin/removecompletedtorrents > /dev/null 2>&1

StdOut

When you run a process on the command line, it will probably output right back to the terminal. This is StdOut, or “standard output.” In scripts and cron jobs, however, you might want to redirect output to a file instead, so you can review the output later.

You can redirect output to a new file by using the “>” operator.

$ ls -l > ~/directory_listing.txt

You can redirect output and append it to a file by using the “>>” operator. This method will create a file if it does not already exist.

StdErr

Error messages are output to StdErr, which by default is redirected to StdOut. This is why you see normal messages and error messages in the terminal when you run a command. StdErr does not have to be redirected to StdOut, however, and has to be treated separately from StdOut if you are redirecting StdOut.

/dev/null and 2>&1

Redirecting output to /del/null sends it nowhere, which effectively gets rid of it. You often see output redirected to /dev/null, followed by a cryptic construction 2>&1. Here is what that construction means:

In the command
$ tvnamer --batch > ~/log/tvnamer.log 2>&1

• 2 refers to StdErr; and
• &1 refers to the first redirection argument; in this case, /log/tvnamer.log.

My example uses a normal file, instead of /dev/null, to show that you can use  2>&1 with any old file.

This construction is very useful if you find that your cron jobs are sending you emails, even after you redirected their output to log files. Cron will automatically email the owner of the cron job any output (on StdOut) from them. This is why you sometimes are notified that you have emails when you log into your system. Effective redirection will eliminate these emails.

This script uses command line utility transmission-remote (which is installed when you install the transmission-daemon package) to check on the progress of all active torrents. Torrents that have completed downloading and seeding will have a status of “Stopped” and a (download) percent done of “100%”. This script will identify such torrents, move their downloaded files to a different directory, and will remove the torrent from transmission-daemon.

This script depends on using a seeding ratio to stop your torrents from seeding after a certain percentage has been seeded. If your torrents seed indefinitely, transmission-daemon will never stop them, and this script will not know that the torrent is completed.

Note that transmission-daemon has a built-in option to store incomplete downloads in one folder, and move them, when the download completes, to another folder. This script is meant to pick the torrents up afterward, and move them from the download folder to somewhere more permanent. If you just want to remove completed torrents, comment out the line “transmission-remote --torrent $TORRENTID --netrc $NETRC --move $MOVEDIR“.

Script: movecompletedtorrents

This script is a modified version of the one found at http://forum.qnap.com/viewtopic.php?f=45&t=25553.

#!/bin/sh

# script to check for complete torrents in transmission folder, then stop and move them

MOVEDIR=/home/user/media
NETRC=/home/user/.netrc

# get torrent list from transmission-remote list
# delete first / last line of output
# remove leading spaces
# get first field from each line
TORRENTLIST=`transmission-remote --list --netrc $NETRC | sed -e '1d;$d;s/^ *//' | cut -s -d " " -f1`

# for each torrent in the list
for TORRENTID in $TORRENTLIST
do
  echo "* * * * * Operations on torrent ID $TORRENTID starting. * * * * *"

  # check if torrent was started
  STARTED=`transmission-remote --torrent $TORRENTID --info --netrc $NETRC | grep "Id: $TORRENTID"`
  # echo " - started state = $STARTED" # debug message

  # check if torrent download is completed
  COMPLETED=`transmission-remote --torrent $TORRENTID --info --netrc $NETRC | grep "Percent Done: 100%"`
  # echo " - completed state = $COMPLETED" # debug message

  # check torrent's current state is "Stopped"
  STOPPED=`transmission-remote --torrent $TORRENTID --info --netrc $NETRC | grep "State: Stopped"`
  # echo " - torrent stopped seeding = $STOPPED" # debug message

  # if the torrent is "Stopped" after downloading 100% and seeding, move the files and remove the torrent from Transmission
  if [ "$STARTED" != "" ] && [ "$COMPLETED" != "" ] && [ "$STOPPED" != "" ]; then
    echo "Torrent #$TORRENTID is completed."
    echo "Moving downloaded file(s) to $MOVEDIR."
    transmission-remote --torrent $TORRENTID --netrc $NETRC --move $MOVEDIR
    echo "Removing torrent from list."
    transmission-remote --torrent $TORRENTID --netrc $NETRC --remove
  else
    echo "Torrent #$TORRENTID is not completed. Ignoring."
  fi

  echo "* * * * * Operations on torrent ID $TORRENTID completed. * * * * *"

done

Scheduling

To run this script every hour, save the script somewhere (/home/user/bin), grant yourself permission to execute it…

$ chmod u+x ~/bin/movecompletedtorrents

…edit your crontab settings…

$ crontab -e

…and add the following line.

@hourly /home/user/bin/movecompletedtorrents

How It Works

I thought I would have to write a complex regular expressions script to reformat the junky file names I end up with. TVNamer does this behind the scenes, and goes one very useful step beyond that: it compares the file names you pass it to the TVDB database, via the TVDB API, to ensure you get a good match. The TVDB is an open database that stores the show titles, episode numbers, and so on, for nearly any show you can think of. As long as your file name contains the series title (in some format or another), and the season and episode numbers (S01E01, for example), TVNamer will rename them nicely.

Installation

TVNamer requires Python 2.6. If you don’t have it, install it as follows:

apt-get install python2.6

The tvnamer program is simple to install on Ubuntu (I run Ubuntu Server 10.04 Lucid Lynx) via Python’s “easy_install” command. (This probably works on all platforms that Python runs on, too.)

$ sudo easy_install tvnamer

Running Manually

You can run tvnamer on one file…

$ tvnamer seriesname.s01e01.crap.you.do.not.care.about.in.the.file.name.mkv

…or on an entire folder.

$ tvnamer ~/media/tv/

It can also recursively run on any folder tree that you pass to it. I have found this to be the default behavior, though, so the –recursive argument does not appear to be necessary. (I tend to use the argument in my cron jobs, in case future versions of tvnamer start to require it.)

$ tvnamer --recursive ~/media/tv/

Note that when you run the program as written above, it will run in interactive mode. For each file you pass to it, you will have to verify the series name and verify that you wish to rename the file.

Running Automatically on a Schedule

Fortunately, you can also run tvnamer in batch mode. Batch mode will not prompt the user at all, and will automatically select the most-relevant series title and will automatically rename each file. This is the mode to run tvnamer via a cron job or a shell script.

Batch mode:

$ tvnamer --batch ~/media/tv/

To schedule a cron job to run this script hourly, run:

$ crontab -e

and then add the following line to your crontab:

@hourly tvnamer --batch --recursive ~/media/tv/

and finally save the crontab file.