Jump to content

System Monitoring with Xymon/Other Docs/HOWTO

Add links
From Wikibooks, open books for an open world

How do I clone maintenance records from one xymon to the other one ?

[edit | edit source]

The front-end web GUI is to collect information and in the end send out bb command with "disable" syntax.

Found out the maintenance status of a host

[edit | edit source]
  • bb xymon-server-name "hobbitdboard color=blue fields=hostname,testname,disabletime,dismsg"

Disable a host

[edit | edit source]
  • bb xymon-server-name "disable hostname.testname $timeframe $REASON"

Enable a host

[edit | edit source]
  • bb xymon-server-name "enable hostname.testname"

Enable All hosts with blue record that in maintenance mode

[edit | edit source]

Why ?, Useful when you need to populate the records from hobbit server A to B. 

BB=/usr/bin/bb
HBS=myhobbit.example.com
${BB} ${HBS}  "hobbitdboard color=purple fields=hostname,testname" |
while read L; do
      HOSTANDTEST=`echo $L | sed 's/|/./'`
      ${BB} ${HBS} "enable $HOSTANDTEST"
done

Replicate blue record from Xymon A to Xymon B

[edit | edit source]
#! /bin/sh
# -----------------------------------------------------------------------------
#           S H E L L  S C R I P T   S P E C I F I C A T I O N
# -----------------------------------------------------------------------------
#
# NAME
#      bluesync.sh  - A shell script to replicate Primary Xymon Serverblue record
#                     to Secondary standby Xymon server.
#
# REVISION HISTORY
#      07/12/2009    Base on getblue.sh and putblue.sh by Ralph Mitchell <ralphmitchell (at) gmail.com>
#                    http://www.hswn.dk/hobbiton/2009/07/msg00214.html
#      10/19/2009    T.J. Yang merge two scripts into one for Xymon  High Availability setup.
#
# USAGE
#      This script is tested on Solaris 10 OS.
#      put this shell script in hobbitlaunch.cfg as section like following.
#
# [bluesync]
# #     DISABLED
#       ENABLE
#       ENVFILE /etc/opt/hobbitserver42/hobbitserver.cfg
#       CMD /opt/hobbitserver42/ext/bluesync/bluesync.sh
#       LOGFILE $BBSERVERLOGS/bluesync.sh.log
#       INTERVAL 5m
#
# DESCRIPTION
#     Xymon keep its blue(maintenance) in memory, not in a file. This script
#     is to copy the blue records from primary Xymon A server into Xymon B secondary one.
#     This script is for use when running Active-Active type of Xymon server
#     High Availability setup. Xymon B is same as Xymon A except
#     the alerting function is disabled. There is another script running on Xymon B to
#     detect the outage of Xymon A and enable Xymon B to become primary one.
#
# Algorithms:
# 1. Pull blue records from Xymon1 and dump them into Xymon2's ext/bluesync directory.
# 2. Process the lifetime fields' value
#     if lifetime is -1 and greater then zero
#       then run "bb hobbit2.test.com "disable hosname.test lifetime message"
#            to keep the blue record.
#     if  liftime =0
#       the run "bb hobbit2.test.com "enable  hostname.*d"  to enable this host blue record
#           since it is expired or was enabled on xymon1 server.
#

# RETURN CODE
#       SUCCESS (=0) - script completed successfully
#       ERROR   (=1) - error... bad things happened
#       WARNING (=2) - warning... something's not quite right, but it's
#                      not serious enough to prevent installation.
#
# set -x

# ---------------------------- CONSTANT DECLARATION ---------------------------
XYMONCFG="/etc/opt/hobbitserver42/hobbitserver.cfg"
BlueTxt="/opt/hobbitserver42/ext/bluesync/hobbit1.test.com.blue.txt"
HB1="hobbit1.test.com"
SED="/usr/bin/sed"
HB2="hobbit2.test.com"
NOW=`/opt/bin/gdate +%s`
EXPR="/usr/bin/expr"
RM="/usr/bin/rm"
SUCCESS=0
ERROR=1
WARNING=2
export SUCCESS ERROR WARNING
# inherit hobbit server variables in configuration file
. ${XYMONCFG}

# ---------------------------- VARIABLE DECLARATION ---------------------------
exit_code=${SUCCESS}

# ******************************** MAIN SCRIPT ********************************

# remove the old file first.
${RM} ${BlueTxt}
${exit_code} = $?

# Getting blue records into a text file.

if [ ${exit_code} -eq ${SUCCESS} ]; then
   $BB ${HB1} "hobbitdboard color=blue fields=hostname,testname,disabletime,dismsg" |
           ${SED} -e 's/\\/\\\\/g'  > ${BlueTxt}
${exit_code} = $?
fi

if [ ${exit_code} -eq ${SUCCESS} ]; then
    cat ${BlueTxt} | while read line
    do
      OFS="$IFS"
      IFS="|"
      set $line
      IFS="$OFS"
      if [ "$3" -eq "-1" ]; then
        # found "disable until OK"
        lifetime="-1"
      else
        lifetime=`$EXPR $3 - ${NOW}`
        if [ "$lifetime" -le "0" ]; then
          # this one expired, enable it
          $BB ${HB2} "enable $1.*"
        else
          lifetime="$lifetime"s
        fi
      fi
      msg=`echo "$4" | $SED -e 's/\\\n/\n/g'`
      $BB ${HB2} "disable $1.$2 $lifetime $msg"
    done
${exit_code} = $?
fi

exit ${exit_code}

How to have pca work with xymon ?

[edit | edit source]

pca is a perl script that can report and install patch missing patches from Sun's patches databases for all versions of Solaris OS.

There are two ways to integrate pca with xymon monitoring tool.

Server side patch analysis

[edit | edit source]

Run "showrev -p" and "pkginfo -x" on xymon client and run pca.pl on xymon server to analyze missing patch for all the clients.

Client side patch analysis

[edit | edit source]

Run pca.pl on xymon client for missing patches analysis and send in lists of missed patch to xymon server.

  • pca software and its dependent software need to be deployed as part of xymon client package.
  • pca.pl need to be configure to query internal Solaris patch database reside on xymon server.
  • Need to set up a local pca proxy server to store all the patch zip files and main solaris patch database.
  • Use pca command to update missing patch after getting approvals to update the missing patches.

Pros and Cons for Server and Client side methods

[edit | edit source]
Integration approach Server side analysis Client Side analysis
Client changes needed Modify -sunos.sh to send in "showrev -p" and "pkginfo -x" output. Run "pca -L" and more HTML code correction in wrapper script or run "pca -M" (with a patch change to pca.pl) on Xymon client as a client side ext module.
Server side change needed Write a server side module to process the data of "showrev -p" sent from clients. Xymon server will create a "pca" column with missed patch send from Xymon client.
Load on Xymon server This will add the missing patch calculation time for all clients on the Xymon server. Load to xymon server is minimum because the calculation task is done on Xymon clients.
pca script changes No pca.pl script change is needed. We only update the new pca.pl one time on the Xymon server. "pca -L" won't generate Xymon client compatible HTML code. We need to either correct this in the wrapper script or patch the pca perl code.
client to server traffic The raw data output of "showrev -p" will be sent every 5 minutes! The message size is only the missed patches list, not the whole output of "showrev -p" and every 5 minutes.
Administration Cost TBD TBD

Transparent Authentication against Active Directory 2003 with Apache and CentOS 5

[edit | edit source]

Here, I (Stewart L) will explain the steps I went through to get a Linux server joined to our Active Directory 2003 infrastructure and to authenticate users against the domain without them being required to enter credentials.

This is against an AD 2003 structure. If you are operating in a 200 or NT domain, this might not work for you, but it should point you on your way.

I'll make a few assumptions at this point for the example.

  • You are setting up a webserver to be named web1.example.com.
  • Your domain is called EXAMPLE and your kerberos Realm is named EXAMPLE.COM
  • You have a domain account named EXAMPLE\Bob that is authorized to add machines into the domain.
  • Your Domain controller is dc1.example.com.

Install Packages

[edit | edit source]

You obviously need Apache installed. You will also need the mod_auth_kerb package to authenticate against the domain. It is also much easier if you use the system-config-authentication tool in the authconfig-gtk package. 

yum -y install mod_auth_kerb authconfig-gtk

Join the Machine to the Domain

[edit | edit source]

Before you can join a machine to a domain, you must have a few items taken care of...

  • The hostname (excluding the domain) should be 15 characters or less.
  • The system clocks should be synchronized. Use NTP for this.
  • Your /etc/hosts file needs to be properly set up. You should have a localhost entry pointing to 127.0.0.1 and an entry that has your fully-qualified host name pointing to its assigned IP address.

With that out of the way, we can begin configuring authentication.

  1. Run system-config-authentication as root.
  2. On the Authentication tab, Enable Kerberos and Winbind
  3. Configure Kerberos.
    1. REALM = EXAMPLE.COM
    2. Check the boxes for using DNS to resolve hosts to realms and locate KDCs.
    3. KDC and Admin Sevrer can be left blank
    4. Click ok.
  4. Configure Winbind
    1. Domain = EXAMPLE
    2. Security Model = ads
    3. ADS Realm = EXAMPLE.COM
    4. Domain Controllers = dc1.example.com
    5. Click ok
  5. Edit your /etc/samba/smb.conf file and make sure that your netbios name is the same as your hostname. This should be the host part only, not the domain.

Join the Domain

[edit | edit source]

As the root user, run the following commands. You will have to enter a password for Bob after both commands. 

kinit EXAMPLE\Bob
net ads join -U EXAMPLE\Bob

This is the domain. By default you have to have a local account on the box to authenticate against AD, meaning if there is not a bob account on web1.example.com, bob cannot log in with his domain password.

Configure an AD User

[edit | edit source]

This is where things become a little convoluted. We are going to create a user account in AD that the web server will use for authentication. There are a number of different versions and service packs out there for Windows Server 2000 and 2003. Check http://grolmsnet.de/kerbtut/ if you have problems with this part.

  • Create a user in AD named http_web1.
  • Set this account so that the password never expires.
  • On the command line of the Domain Controller, run this line
ktpass -princ HTTP/web1.example.com@EXAMPLE.COM -mapuser EXAMPLE\http_web1 -crypto DES-CBC-MD5 -ptype KRB5_NT_SRV_HST -pass * -out c:\temp\http_web1.heytab
  • This will create a keytab file in C:\temp that you need to move to your webserver and place in /etc/http.

Configure Apache

[edit | edit source]

Your configuration should look something like this... 

<Location />
   AuthName "Welcome to EXAMPLE"
   AuthType Kerberos
   Krb5Keytab /etc/httpd/http_web1.keytab
   KrbAuthRealm EXAMPLE.COM
   KrbMethodNegotiate On
   KrbSaveCredentials off
   KrbVerifyKDC off
   Require valid-user
   </Location>

It is possible to change the Authname a new name. Check http://modauthkerb.sourceforge.net/ for more info on specific configurations.

Configure Firefox (Optional)

[edit | edit source]
  1. Type about:config in the URL bar
  2. Modify the following "Preference Name"
    • network.negotiate-auth.delegation-uris - Example.com
    • network.negotiate-auth.trusted-uris - Example.com
    • network.automatic-ntlm-auth.trusted-uris - Example.com

How do I prevent red/purple alert storm from Xymon server ?

[edit | edit source]

What

[edit | edit source]
  • red alert storm:
    • caused by death of a few switches and bad bb-hosts configuration that missing router dependency directive.
    • human error on configuring fping access control list.
  • purple storm:
    • caused by Xymon server went into very high CPU load itself that prevent itself from running the Xymon daemon themselves properly thus all machines went into purple due lack of message update within 30 minutes.

How to fix

[edit | edit source]
  • Deploy mail filter to restrict the volume of alerts Xymon server can send out.

How do I update xymon client's configuration file from xymon server ?

[edit | edit source]

Why ?

[edit | edit source]
  • To make global variable changes on all hobbit clients.
    • Examples: To configure all your clients to send hobbit messages from one to two Xymon servers. This will need to modify hobbitclient.cfg on all clients. If you have small pool of clients, manually editing by login into each machines is ok. But for a pool of 3000 clients, this change is better done via clientudpate or cluster shell plus rsh approach.
    • This wiki page is to verify if xymon's clientupdate really work for this need.

What need to be done ?

[edit | edit source]
  • Update hobbit-client.cfg on hobbit clients with new hobbit-client.cfg locate in central hobbit server. The new hobbit-client.cfg contain new configuration to send hobbit messages to two hobbit server.
    • diff of old and new hobbit-client.cfg file.
bash-2.05$ diff hobbitclient.cfg hobbitclient.cfg.new
3,4c3,4
< BBDISP="hobbit.test.com"         # IP address of the Hobbit server
< BBDISPLAYS=""                   # IP of multiple Hobbit servers. BBDISP must be "0.0.0.0".
---
> BBDISP="0.0.0.0"      # IP address of the Hobbit server
> BBDISPLAYS="hobbit.test.com hobbit2.test.com"  # IP of multiple Hobbit servers. BBDISP must be "0.0.0.0".
bash-2.05$
  • Make sure/Modify all the scripts in ext directory send messages to BBDISPLAYS.
    • temperature module.
      • original
## Report the data to the $BBDISP host
if [ "$TEMPDEBUG" = 1 ]; then
        echo "$DATA"                            # Send it to stdout
else
        $BB $BBDISP "$LINE"                     # SEND IT TO BBDISPLAY
fi
      • changes for two hobbit servers.
## Report the data to the $BBDISP host
if [ "$TEMPDEBUG" = 1 ]; then
        echo "$DATA"                            # Send it to stdout
else
    if [ "$BBDISP" = "0.0.0.0" ];then
        for i in "$BBDISPLAYS"
        do
        $BB $i "$LINE"                     # SEND IT TO BBDISPLAYS
        done
    else
        $BB $BBDISP "$LINE"                     # SEND IT TO BBDISPLAY
fi

RTFM first

[edit | edit source]

Procedures

[edit | edit source]

To manage updating clients without having to logon to each server, you can use the clientupdate utility. This is how you setup the release of a new client version.

  1. make sure the xymon client external scripts send message to "BBDISPLAYS", not just "BBDISP" variable.
    1. this require the review of external(home-grown) script got deployed to be modified to send messages to two xymon servers.
  2. Create the new client: Setup the new client $BBHOME directory, e.g. by copying an existing client installation to an empty directory and modifying it for your needs. It is a good idea to delete all files in the tmp/ and logs/ directories, since there is no need to copy these over to all of the clients. Pay attention to the etc./ files, and make sure that they are suitable for the systems where you want to deploy this new client. You can add files - e.g. extension scripts in the ext/ directory - but the clientupdate utility cannot delete or rename files.
  3. Package the client: When your new client software is ready, create a tar-file of the new client. All files in the tar archive must have filenames relative to the clients' $BBHOME (usually, ~hobbit/client/). Save the tarfile on the Hobbit server in ~hobbit/server/download/somefile.tar. Don't compress it. It is recommended that you use some sort of operating-system and version-numbering scheme for the filename, but you can choose whatever filename suits you - the only requirement is that it must end with ".tar". The part of the filename preceding ".tar" is what Hobbit will use as the "clientversion" ID.
  4. Configure which hosts receive the new client: In the client-local.cfg(5) file, you must now setup a clientversion:ID line where the ID matches the filename you used for the tar-file. So if you have packaged the new client into the file linux.v2.tar, then the corresponding entry in client-local.cfg would be clientversion:linux.v2.
  5. Wait for hobbitd to reload client-local.cfg: hobbitd will automatically reload the client-local.cfg file after at most 10 minutes. If you want to force an immediate reload, send a SIGHUP signal to the hobbitd process.
  6. Wait for the client to update: The next time the client contacts the Hobbit server to send the client data, it will notice the new clientversion setting in client-local.cfg, and will run clientupdate to install the new client software. So when the client runs the next time, it will use the new client software.

Getting started with clientupdate

[edit | edit source]
  • find out the hobbiclient version, we are running 4.2.0 from output of "clientupdate --level"
 bash-3.00$ ./bbcmd
 2009-01-28 20:50:39 Using default environment file /etc/opt/hobbitclient42/hobbitclient.cfg
 bash-3.00$ ./clientupdate --level
 4.2.0
 bash-3.00$
    • or cutout the version from clientversion.cfg.
bash-3.00$ cat  clientversion.cfg
4.2.0
bash-3.00$
  • do the update but no new version on server side.
bash-3.00$ ./clientupdate
2009-01-28 20:53:25 No new version string!
bash-3.00$

Error messages from clientupdate

[edit | edit source]
  • core dump because we didn't run bbcmd first. clientupdate.c need to be updated to avoid the coredump.
bash-3.00$ ./clientupdate
2009-01-28 20:49:13 xgetenv: Cannot find value for variable ETCBBHOME
Segmentation Fault (core dumped)
bash-3.00$
  • "--suid-setup failed: No such file or directory", no xymon server side configuration at all.
bash-2.05$ ./clientupdate --update=4.2.1
tar: blocksize = 0
2009-01-28 16:03:39 exec() of clientupdate --suid-setup failed: No such file or directory
bash-2.05$

How do I configure sudo to empower hobbit client to gain limited root privilege ?

[edit | edit source]

Add SUDO path to hobbit client main script

[edit | edit source]

We need to modify hobbitclient.sh$.{OS} to use sudo without prompting password on root privileged access. Following is an patch for Linux client. you need to do similar patching for other OS types. 

[root:myhbserver] cat  src/hobbitclient.sh.linux.patch
--- client/hobbitclient.sh.orig 2008-04-12 08:41:20.050938000 -0500
+++ client/hobbitclient.sh      2008-04-12 08:41:26.224999984 -0500
@@ -18,7 +18,16 @@
 LANG=C
 LC_ALL=C
 LC_MESSAGES=C
-export LANG LC_ALL LC_MESSAGES
+# Why : Make hobbit client to be non-intrusive and avoid chmod system file
+# What: use sudo to fetch /var/log/messages on linux in 600 file mode
+# # Rules for HOBBIT client
+# logfech will failed on Linux because /var/log/messages is default to 600
+# User_Alias HOBBITCLIENT = hobbitc
+# Cmnd_Alias HOBBITCLIENTCMDS = /usr/sbin/swapinfo,/usr/sbin/vxdisk,/usr/bin/ipmitool,/opt/bin/logfetch
+# HOBBITCLIENT ALL = NOPASSWD: HOBBITCLIENTCMDS
+
+SUDO=/opt/bin/sudo
+export LANG LC_ALL LC_MESSAGES SUDO

 LOCALMODE="no"
 if test $# -ge 1; then
@@ -53,7 +62,7 @@
 # logfiles
 if test -f $LOGFETCHCFG
 then
-    $HOBBITCLIENTHOME/bin/logfetch $LOGFETCHCFG $LOGFETCHSTATUS >>$MSGTMPFILE
+    ${SUDO} $HOBBITCLIENTHOME/bin/logfetch $LOGFETCHCFG $LOGFETCHSTATUS >>$MSGTMPFILE
 fi
 # Client version
 echo "[clientversion]"  >>$MSGTMPFILE
[root:myhbserver]

configure your sudoer file

[edit | edit source]

hobbits = hobbit server user account. hobbitc = hobbit client account. 

[root:myhbserver] cat /opt/sudo16/etc/sudoers  |grep -v ^$ |grep -v ^#
root    ALL=(ALL) ALL
User_Alias HOBBITUSER = hobbits
Cmnd_Alias HOBBITSVERTCMDS = /opt/fping24/sbin/fping
HOBBITUSER ALL = NOPASSWD: HOBBITSVERTCMDS
User_Alias HOBBITUSER = hobbitc
Cmnd_Alias HOBBITCLIENTCMDS = /usr/sbin/swapinfo,/usr/sbin/vxdisk
HOBBITUSER ALL = NOPASSWD: HOBBITCLIENTCMDS
[root:myhbserver]

To determine whether the host is not configured for alerting

[edit | edit source]

no-alerts.pl listing

[edit | edit source]
#! /usr/bin//perl
##########################################################################################
#
# no-alerts.pl - This script uses bbhostgrep to get a list of sevrers, then fetches their
#       info test. It then parses this looking for 'No Alerts Defined' It then outputs the
#       hostname of the machines that are missing alert config.
#
#  Created by Stewart Larsen
#  06/02/2008: minor editing T.J. Yang
##########################################################################################

use strict;
use LWP::Simple;

# Set debug to 1 to see the hosts and URL as you loop through.
my $debug = 0;

#Token in bb-hosts to indicate we need to run firewall tests on this device my $test_name = '*';
##########################################################################################
# Constant:
##########################################################################################
my $CUT  = "/bin/cut";
my $SORT = "/bin/sort";
my $UNIQ = "/usr/bin/uniq";
#ip/domain : server to poll for information
my $BBDISP = "hobbit.test.com";
# hobbit server  install directory
my $HOBBITHOME ="/opt/hobbitserver42";
my $BBHOSTGREP = "$HOBBITHOME/bin/bbhostgrep";

##########################################################################################
# Variables:
##########################################################################################

# Paths - Fetch all hosts and trim for just the host name.
# Then sort and unique the list

my $HOSTGREPOUT="$BBHOSTGREP '*'| $CUT -d' ' -f2 | $SORT  | $UNIQ ";

##########################################################################################
# Main Program:
##########################################################################################

open (INPUT, "-|",$HOSTGREPOUT ) or die "Cannot open input: $!\n";
while(<INPUT>)
{
        chomp (my $host = $_);
        my $fetch_URL = 'http://'.$BBDISP.'/hobbit-cgi/bb-hostsvc.sh?HOST='.$host.'&SERVICE=info';
        print "$host: $fetch_URL\n" if $debug;

        my $content = get $fetch_URL;
        if (!defined $content)
        {
                warn "Couldn't get $fetch_URL";
                next;
        }

        if($content =~ m/No Alerts Defined/i)
        {
                print "$host has no alerts defined\n";
        }
}
close INPUT;

How can I monitor and graph Flexlm license usage ?

[edit | edit source]

The script

[edit | edit source]

The server side rrd configuration file

[edit | edit source]
[flexlm]
TITLE License counts 
YAXIS #

DEF:matlab=flexlm.rrd:MATLAB:AVERAGE
LINE2:matlab#FF0000:Matlab
COMMENT:\n
GPRINT:matlab:LAST:Matlab \: %5.1lf%s (cur)
GPRINT:matlab:MAX: \: %5.1lf%s (max)
GPRINT:matlab:MIN: \: %5.1lf%s (min)
GPRINT:matlab:AVERAGE: \: %5.1lf%s (avg)\n

DEF:simulink=flexlm.rrd:SIMULINK:AVERAGE
LINE2:simulink#CC3333:Simulink
COMMENT:\n
GPRINT:simulink:LAST:Simulink \: %5.1lf%s (cur)
GPRINT:simulink:MAX: \: %5.1lf%s (max)
GPRINT:simulink:MIN: \: %5.1lf%s (min)
GPRINT:simulink:AVERAGE: \: %5.1lf%s (avg)\n

DEF:commtlb=flexlm.rrd:CommunicationToolb:AVERAGE
LINE2:commtlb#66CC66:Communication Toolb
COMMENT:\n
GPRINT:commtlb:LAST:Communication Toolb \: %5.1lf%s (cur)
GPRINT:commtlb:MAX: \: %5.1lf%s (max)
GPRINT:commtlb:MIN: \: %5.1lf%s (min)
GPRINT:commtlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:ctrltlb=flexlm.rrd:ControlToolbox:AVERAGE
LINE2:ctrltlb#6666CC:Control Toolbox
COMMENT:\n
GPRINT:ctrltlb:LAST:Control Toolbox \: %5.1lf%s (cur)
GPRINT:ctrltlb:MAX: \: %5.1lf%s (max)
GPRINT:ctrltlb:MIN: \: %5.1lf%s (min)
GPRINT:ctrltlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:fintlb=flexlm.rrd:FinancialToolbox:AVERAGE
LINE2:fintlb#00FF00:Financial Toolbox
COMMENT:\n
GPRINT:fintlb:LAST:Financial Toolbox \: %5.1lf%s (cur)
GPRINT:fintlb:MAX: \: %5.1lf%s (max)
GPRINT:fintlb:MIN: \: %5.1lf%s (min)
GPRINT:fintlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:fztlb=flexlm.rrd:FuzzyToolbox:AVERAGE
LINE2:fztlb#66FF66:Fuzzy Toolbox
COMMENT:\n
GPRINT:fztlb:LAST:Fuzzy Toolbox \: %5.1lf%s (cur)
GPRINT:fztlb:MAX: \: %5.1lf%s (max)
GPRINT:fztlb:MIN: \: %5.1lf%s (min)
GPRINT:fztlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:imgtlb=flexlm.rrd:ImageToolbox:AVERAGE
LINE2:imgtlb#0000FF:Image Toolbox
COMMENT:\n
GPRINT:imgtlb:LAST:Image Toolbox \: %5.1lf%s (cur)
GPRINT:imgtlb:MAX: \: %5.1lf%s (max)
GPRINT:imgtlb:MIN: \: %5.1lf%s (min)
GPRINT:imgtlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:compiler=flexlm.rrd:Compiler:AVERAGE
LINE2:compiler#6666FF:Compiler
COMMENT:\n
GPRINT:compiler:LAST:Compiler \: %5.1lf%s (cur)
GPRINT:compiler:MAX: \: %5.1lf%s (max)
GPRINT:compiler:MIN: \: %5.1lf%s (min)
GPRINT:compiler:AVERAGE: \: %5.1lf%s (avg)\n

DEF:neuralnettlb=flexlm.rrd:NeuralNetworkTool:AVERAGE
LINE2:neuralnettlb#FFFF00:Neural Network Tool
COMMENT:\n
GPRINT:neuralnettlb:LAST:Neural Network Tool \: %5.1lf%s (cur)
GPRINT:neuralnettlb:MAX: \: %5.1lf%s (max)
GPRINT:neuralnettlb:MIN: \: %5.1lf%s (min)
GPRINT:neuralnettlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:opttlb=flexlm.rrd:OptimizationToolbo:AVERAGE
LINE2:opttlb#00FFFF:Optimization Toolbo
COMMENT:\n
GPRINT:opttlb:LAST:Optimization Toolbo \: %5.1lf%s (cur)
GPRINT:opttlb:MAX: \: %5.1lf%s (max)
GPRINT:opttlb:MIN: \: %5.1lf%s (min)
GPRINT:opttlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:distributedtlb=flexlm.rrd:DistribComputingT:AVERAGE
LINE2:distributedtlb#CC66CC:Distrib Computing T
COMMENT:\n
GPRINT:distributedtlb:LAST:Distrib Computing T \: %5.1lf%s (cur)
GPRINT:distributedtlb:MAX: \: %5.1lf%s (max)
GPRINT:distributedtlb:MIN: \: %5.1lf%s (min)
GPRINT:distributedtlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:pdetlb=flexlm.rrd:PDEToolbox:AVERAGE
LINE2:pdetlb#CCCC66:PDE Toolbox
COMMENT:\n
GPRINT:pdetlb:LAST:PDE Toolbox \: %5.1lf%s (cur)
GPRINT:pdetlb:MAX: \: %5.1lf%s (max)
GPRINT:pdetlb:MIN: \: %5.1lf%s (min)
GPRINT:pdetlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:signaltlb=flexlm.rrd:SignalToolbox:AVERAGE
LINE2:signaltlb#66CCCC:Signal Toolbox
COMMENT:\n
GPRINT:signaltlb:LAST:Signal Toolbox \: %5.1lf%s (cur)
GPRINT:signaltlb:MAX: \: %5.1lf%s (max)
GPRINT:signaltlb:MIN: \: %5.1lf%s (min)
GPRINT:signaltlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:simulctrl=flexlm.rrd:SimulinkControlDe:AVERAGE
LINE2:simulctrl#CCCC00:Simulink Control
COMMENT:\n
GPRINT:simulctrl:LAST:Simulink Control \: %5.1lf%s (cur)
GPRINT:simulctrl:MAX: \: %5.1lf%s (max)
GPRINT:simulctrl:MIN: \: %5.1lf%s (min)
GPRINT:simulctrl:AVERAGE: \: %5.1lf%s (avg)\n

DEF:stattlb=flexlm.rrd:StatisticsToolbox:AVERAGE
LINE2:stattlb#999966:Statistics Toolbox
COMMENT:\n
GPRINT:stattlb:LAST:Statistics Toolbox \: %5.1lf%s (cur)
GPRINT:stattlb:MAX: \: %5.1lf%s (max)
GPRINT:stattlb:MIN: \: %5.1lf%s (min)
GPRINT:stattlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:symboltlb=flexlm.rrd:SymbolicToolbox:AVERAGE
LINE2:symboltlb#FFCC00:Symbolic Toolbox
COMMENT:\n
GPRINT:symboltlb:LAST:Symbolic Toolbox \: %5.1lf%s (cur)
GPRINT:symboltlb:MAX: \: %5.1lf%s (max)
GPRINT:symboltlb:MIN: \: %5.1lf%s (min)
GPRINT:symboltlb:AVERAGE: \: %5.1lf%s (avg)\n

DEF:idtlb=flexlm.rrd:IdentificationTool:AVERAGE
LINE2:idtlb#FF00FF:Identification Tool
COMMENT:\n
GPRINT:idtlb:LAST:Identification Tool \: %5.1lf%s (cur)
GPRINT:idtlb:MAX: \: %5.1lf%s (max)
GPRINT:idtlb:MIN: \: %5.1lf%s (min)
GPRINT:idtlb:AVERAGE: \: %5.1lf%s (avg)\n

How can I create a hobbit client custom test script?

[edit | edit source]
Main article: System Monitoring with Hobbit/HOWTO/Custom Monitoring Scripts for Hobbit

Anything that can be automated via a script or a custom program can be added into Hobbit. A lot of extension scripts are available for Big Brother at the www.deadcat.net archive, and these will typically work without modifications if you run them in Hobbit. Sometimes a few minor tweaks are needed - the Hobbit mailing list can help you if you don't know how to go about that.

But if you have something unique you need to test, writing an extension script is pretty simple. You need to figure out some things:

  • What name will you use for the column?
  • How will you test it?
  • What criteria should decide if the test goes red, yellow or green?
  • What extra data from the test will you include in the status message?

A simple client-side extension script looks like this: 

#!/bin/sh
 
COLUMN=mytest	# Name of the column
COLOR=green		# By default, everything is OK
MSG="Bad stuff status"

# Do whatever needed to test for something
# As an example, go red if /tmp/badstuff exists.
if test -f /tmp/badstuff
then
   COLOR=red
   MSG="${MSG}

   `cat /tmp/badstuff`
   "
else
   MSG="${MSG}

   All is OK
   "
fi

# Tell Hobbit about it
$BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date` ${MSG}"

exit 0

You will notice that some environment variables are pre-defined: BB, BBDISP, MACHINE are all provided by Hobbit when you run your script via hobbitlaunch. Also note how the MSG variable is used to build the status message - it starts out with just the "Bad stuff status", then you add data to the message when we decided what the status is.

To run this, save your script in the ~hobbit/client/ext/ directory (i.e. in the ext/ directory off where you installed the Hobbit client), then add a new section to the ~hobbit/client/etc/clientlaunch.cfg (or ~hobbit/server/etc/hobbitlaunch.cfg on the server) file like this: 

[myscript]
   ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg
   CMD $HOBBITCLIENTHOME/ext/myscript.sh
   LOGFILE $HOBBITCLIENTHOME/logs/myscript.log
   INTERVAL 5m

Optionally you may want to add a description for the new column your custom script creates. On the hobbit *server* edit the file $BBHOME/columndoc.csv adding a line like: 

myscript;The <b>myscript</b> column shows the status of my custom script.;

How to write server side module ?

[edit | edit source]

Server-side scripts look almost the same as client-side scripts, but they will typically use the bbhostgrep utility to pick out hosts in the bb-hosts file that have a special tag defined, and then send one status message for each of those hosts. Like this: 

#!/bin/sh

BBHTAG=foo           # What we put in bb-hosts to trigger this test
COLUMN=$BBHTAG	# Name of the column, often same as tag in bb-hosts

$BBHOME/bin/bbhostgrep $BBHTAG | while read L
do
   set $L	# To get one line of output from bbhostgrep

   HOSTIP="$1"
   MACHINEDOTS="$2"
   MACHINE='echo $2 | $SED -e's/\./,/g'`

   COLOR=green
   MSG="$BBHTAG status for host $MACHINEDOTS"

   #... do the test, perhaps modify COLOR and MSG

   $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date`

   ${MSG}
   "
done
 
exit 0

Note that for server side tests, you need to loop over the list of hosts found in the bb-hosts file, and send one status message for each host. Other than that, it is just like the client-side tests.

This script is saved in ~/hobbit/server/ext/my_script.sh, and you add its description in ~hobbit/server/etc/hobbitlaunch.cfg: 

[foo]
   ENVFILE $BBTHOME/etc/hobbitserver.cfg
   CMD $BB/ext/my_script.sh
   LOGFILE $BB/logs/myscript.log
   INTERVAL 5m

The service can now be added to the bb-hosts file: 

127.0.0.1 my_server # foo

How can I test a custom script before adding it to the running agent ?

[edit | edit source]

Before activating a new custom script it is useful to check its functionality and output on stdout.

To run a script on the command line, you have to load all the xymon environment and replace the $BB command by "echo".

You can easily do this on-the-fly by inserting this code snippet and then test your script by running it on the command line: 

TEST=1 ./my_script.sh

There is nothing you have to change in your script if you want to add it to your hobbitlaunch.cfg after testing. 

#!/bin/sh
...
# Test-Mode ?
# TEST=0 => works in cooperation with Xymon agent (default)
# TEST=1 => for testing, results to stdout (run "TEST=1 ./bb-testname.sh" on cmd-line)
# TEST>1 => for testing, but sending results to Xymon server without activating script in hobbitlaunch.cfg
TEST=${TEST:-"0"}
if [ "$TEST" -gt "0" ]
then
  BBHOME="$HOME/xymon/server"
  . $BBHOME/etc/hobbitserver.cfg
  BBQUERYCMD=$BB
  [ "$TEST" = "1" ] && BB="echo"
fi
...

This works for client-scripts as well as for server side scripts.

Of course you have to change the path settings to your environment.

Hobbit Client and ZFS monitoring

[edit | edit source]
  • See the ZFS tutorial for a simple ZFS pool example.
  • In this example, we have the 3rd and 4th disks available for zfs. 1st and 2nd are under SVM already.
bash-3.00# metastat |tail -3
Device   Reloc  Device ID
c0t1d0   Yes    id1,sd@SSEAGATE_ST336607LSUN36G_3JA6ECDF00007418PJ5V
c0t0d0   Yes    id1,sd@SSEAGATE_ST336607LSUN36G_3JA6EGFK00007418A7PM
bash-3.00#

bash-3.00# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@1c,600000/scsi@2/sd@0,0
       1. c0t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
          /pci@1c,600000/scsi@2/sd@1,0
       2. c0t2d0 <HITACHI-DK32EJ36NSUN36G-PQ0B-33.92GB>
          /pci@1c,600000/scsi@2/sd@2,0
       3. c0t3d0 <HITACHI-DK32EJ36NSUN36G-PQ0B-33.92GB>
          /pci@1c,600000/scsi@2/sd@3,0
Specify disk (enter its number):

bash-3.00# zpool create -f mypool mirror c0t2d0 c0t3d0
bash-3.00# zpool list
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
mypool                 33.8G   84.5K   33.7G     0%  ONLINE     -
bash-3.00#
  • Mike Rowell <Mike.Rowell@Rightmove.co.uk> has the following ZFS hb external module.
#!/bin/ksh
# Revision History: 
# 1. Mike Rowell <Mike.Rowell@Rightmove.co.uk>, original
# 2. Uwe Kirbach <U.Kirbach@EnBW.com>
# 3. T.J. Yang: add in some comments.

DISKYELL=80
DISKRED=90
TEST="zfs"
DISPCOLOR="green"
FIRST_LINE="zfs - okay"
FIRST_LINE_HEALTH="okay"
FIRST_LINE_CAP="okay"

#What: beautify the page display by html code.
STRING="<table border=0 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Capacity</th></tr>"
#What: a loop to parse output of "zpool list -H" output.
# bash-3.00# zpool list
# NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
# mypool                 33.8G   84.5K   33.7G     0%  ONLINE     -
# bash-3.00# zpool list -H
# mypool  33.8G   84.5K   33.7G   0%      ONLINE  -
# bash-3.00#

/usr/sbin/zpool list -H | while read name size used avail cap health altroot
do
  LINE_COLOR="green"

  if [ "${health}" == "ONLINE" ]; then
    HEALTH_COLOR="green"
  elif [ "${health}" == "DEGRADED" ]; then
    HEALTH_COLOR="yellow"
  elif [ "${health}" == "FAULTED" ]; then
    HEALTH_COLOR="red"
  fi

  cap=`echo ${cap} | cut -d% -f1` 
  if [ ${cap} -lt $DISKYELL ]; then
    CAP_COLOR="green" 
  elif [ ${cap} -gt $DISKYELL ]; then 
    CAP_COLOR="yellow"
  elif [ ${cap} -gt $DISKRED ]; then
    CAP_COLOR="red"
  fi

  if [ "$HEALTH_COLOR" == "red" -o "$HEALTH_COLOR" == "yellow" -o "$CAP_COLOR" == "red" -o "$CAP_COLOR" == "yellow" ]; then
    DISPCOLOR=$COLOR
    LINE_COLOR=$COLOR
  fi

  case $HEALTH_COLOR in
    red)    FIRST_LINE_HEALTH="faulted" ;;
    yellow) FIRST_LINE_HEALTH="degraded" ;;
  esac

  case $CAP_COLOR in
    red)    FIRST_LINE_CAP="full" ;;
    yellow) FIRST_LINE_CAP="nearly full" ;;
  esac

  STRING="$STRING <tr><td>&${LINE_COLOR}</td><td>${name}</td><td>${health}</td><td>${cap}</td></tr>"
done

# What: accumulate the bb message strings.
STRING="$STRING </table><br><br>"
STRING="$STRING`/usr/sbin/zpool status -xv`"
FIRST_LINE="zfs - health: $FIRST_LINE_HEALTH - capacity: $FIRST_LINE_CAP"

# What: Sent out the final bb message to hobbit server.
$BB $BBDISP "status $MACHINE.$TEST $DISPCOLOR `date` $FIRST_LINE $STRING"
  • Testing on hobbit client.
bash-3.00$ export BB=/opt/hobbitclient42/bin/bb
bash-3.00$ export BBDISP="hobbit.my.com"
bash-3.00$ export MACHINE="hobbitclient.my.com"
bash-3.00$ ksh -x  zfs.ksh
+ DISKYELL=80
+ DISKRED=90
+ TEST=zfs
+ COLOR=GREEN
+ STRING=<table border=0 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Capacity</th></tr>
+ read name size used avail cap health altroot
+ /usr/sbin/zpool list -H
+ [ ONLINE == ONLINE ]
+ COLOR=green
+ cut -d% -f1
+ echo 0%
+ [ 0 -lt 80 ]
+ COLOR=green
+ [ green == red -o green == yellow ]
+ STRING=<table border=0 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Capacity</th></tr> <tr><td>&green</td><td>mypool</td><td>ONLINE</td><td>0%</td></tr>
+ read name size used avail cap health altroot
+ STRING=<table border=0 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Capacity</th></tr> <tr><td>&green</td><td>mypool</td><td>ONLINE</td><td>0%</td></tr> </table><br><br>
+ + /usr/sbin/zpool status -xv
STRING=<table border=0 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Capacity</th></tr> <tr><td>&green</td><td>mypool</td><td>ONLINE</td><td>0%</td></tr> </table><br><br>all pools are healthy
+ date
+ /opt/hobbitclient42/bin/bb hobbit.my.com status .zfs  Thu Apr 12 10:05:43 CDT 2007 <table border=0 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Capacity</th></tr> <tr><td>&green</td><td>mypool</td><td>ONLINE</td><td>0%</td></tr> </table><br><br>all pools are healthy
bash-3.00$
  • Check that the data arrives in Hobbit server.

Use a Big Brother script without converting it to hobbit

[edit | edit source]

Run the latest Big Brother client on your hobbit server. Execute the desired script from this client, it will report to hobbit. e.g. using the latest "bb-mrtg.pl" script to monitor your mrtg graphs

Add Custom Graphs

[edit | edit source]

Read the man page for hobbitd_rrd

Add support for a database backend

[edit | edit source]

I haven't done it, but it will be easy to implement. For reporting you just want to know what status changes have occurred, so you simply need to come up with a Hobbit "worker module" that receives messages from the Hobbit "stachg" channel, and stores them in your database.

Worker modules can be written in whatever language you fancy, although I'd advise against shell-scripts for performance reasons. Perl, Python, C, C++, Java ... pick your favourite programming tool.

All the module needs to do is read the messages off it's standard input filehandle, and store them in the database. Each message is formatted like this: 

 @@stachg|timestamp|sender|origin|hostname|testname|expiretime|color|prevcolor|changetime
 <new status message>
 @@

It is easy to pick out the first line and grab all of the needed information from that.

There's a sample module provided with the Hobbit sources, see the "hobbit-4.0-RC4/hobbitd/hobbitd_sample.c" file which illustrates how to build a worker module in C.

Installing Hobbit as client with a BB server

[edit | edit source]

There are a few gotchas if you want to use Hobbit as client with a BB server: the Hobbit client usually gets its configuration from the server, and the standard protocol is different. To make it backwards compatible requires a few workarounds on hobbit client side.

Example for Debian Unix OS, adjust as required otherwise.

Start as root 

  apt-get install gcc make
  apt-get install libpcre3 libpcre3-dev
  useradd -m hobbit
  su - hobbit

We are hobbit! 

  tar zxfv .../hobbit-4.2.0.tar.gz
  cd hobbit-4.2.0
  ./configure --client
Answer client
Answer hobbit
Answer /home/hobbit
Answer 10.0.4.5 (replace with the IP address of your BB server)
  make
  make install
  cd
  chmod +w client/runclient.sh
  vi client/runclient.sh
Add MACHINEDOTS="f.q.d.n", i.e. the client's fqdn
Add --local on the line that starts hobbitlaunch
  vi client/etc/hobbitclient.cfg
Add BBMAXMSGSPERCOMBO=1
  exit

We are root again!

Create /etc/init.d/hobbit as: 

  #!/bin/sh
  su hobbit -c "/home/hobbit/client/runclient.sh $1"
  
  chmod +x /etc/init.d/hobbit
  update-rc.d hobbit defaults
  
  /etc/init.d/hobbit start

Monitor Hobbit clients in a DMZ using reverse SSH tunnels

[edit | edit source]

Note: Other methods for doing this type of thing are documented on the System_Monitoring_with_Xymon/Administration_Guide page.

Operating system: Red Hat Enterprise Linux 5
Hobbit version: Hobbit 4.2.0

In some scenarios you may want to monitor Hobbit clients that are separated from the Hobbit server by a firewall (e.g. Hobbit clients located in a DMZ) and where you cannot open port 1984 inbound from the Hobbit clients to the Hobbit server. An alternative would be to use reverse SSH tunnels, using Padraig Lennon's ssh_tunnels.sh script.

Note that for the purpose of this document, the hobbit user's home directory is assumed to be /usr/lib/hobbit.

Passwordless SSH Authentication

[edit | edit source]

For the hobbit user, set up passwordless SSH authentication between the Hobbit server and the Hobbit clients.

On the Hobbit server, generate a private/public keypair with an empty password: 

# su - hobbit

$ ssh-keygen -t dsa

Generating public/private dsa key pair.
Enter file in which to save the key (/usr/lib/hobbit/.ssh/id_dsa):
Created directory '/usr/lib/hobbit/.ssh'.
Enter passphrase (empty for no passphrase):  <-- Ensure that you create an empty passphrase, i.e. just hit Enter.
Enter same passphrase again:
Your identification has been saved in /usr/lib/hobbit/.ssh/id_dsa.
Your public key has been saved in /usr/lib/hobbit/.ssh/id_dsa.pub.
The key fingerprint is:
xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:x:xx:xx hobbit@servername

This should create the following files in the .ssh directory located in the hobbit user's home directory: 

$ ls -ld /usr/lib/hobbit/.ssh
drwx------ 2 hobbit hobbit 4096 Aug 14 12:16 /usr/lib/hobbit/.ssh

$ ls -lR /usr/lib/hobbit/.ssh
/usr/lib/hobbit/.ssh:
total 12
-rw------- 1 hobbit hobbit  668 Aug 14 12:14 id_dsa
-rw-r--r-- 1 hobbit hobbit  603 Aug 14 12:14 id_dsa.pub

If for some reason the creation of the keypair fails, try manually creating the .ssh folder in the hobbit user's home directory before generating the keypair. Ensure that the folder is owned by the hobbit user and group, and that its permissions are correct: 

# chown hobbit:hobbit /usr/lib/hobbit/.ssh
# chmod 700 /usr/lib/hobbit/.ssh

Now you want to copy the generated public key (id_dsa.pub) to the Hobbit client. Ensure that the .ssh folder exists for the hobbit user on the Hobbit client with the same permissions as above, and then copy id_dsa.pub to the Hobbit client: 

$ scp /usr/lib/hobbit/.ssh/id_dsa.pub hobbit@hobbitclient:/usr/lib/hobbit/

On the Hobbit client:

Rename the public key copied from the Hobbit server from "id_dsa.pub" to "authorized_keys": 

$ mv /usr/lib/hobbit/id_dsa.pub /usr/lib/hobbit/authorized_keys

Move the authorized_keys file to the hobbit user’s .ssh directory: 

$ mv /usr/lib/hobbit/authorized_keys /usr/lib/hobbit/.ssh/

Check that the authorized_keys file has the correct permissions (chmod 644): 

$ ls -l /usr/lib/hobbit/.ssh/authorized_keys
-rw-r--r--  1 hobbit hobbit 603 Aug 14 12:20 /usr/lib/hobbit/.ssh/authorized_keys

From the server, SSH to the client. You should add the client's host key to the server's known_hosts file when prompted, but should not be prompted for a password.

Configure the Hobbit client

[edit | edit source]

Add the following line to /usr/lib/hobbit/client/etc/hobbitclient.cfg on the Hobbit client: 

BBDISP="127.0.0.1"

Configure /etc/default/hobbit-client as follows: 

HOBBITSERVERS="127.0.0.1"

Restart the hobbit-client service on the Hobbit client.

Configure the Hobbit server

[edit | edit source]

Copy the ssh-tunnels.sh script to the following location on the Hobbit server, and make sure that its permissions are correct (chmod 755): 

-rwxr-xr-x 1 root root 5469 Aug 14 12:30 /usr/lib/hobbit/server/ext/ssh-tunnels.sh

Modify /etc/hobbit/hobbitlaunch.cfg: 

[ssh-tunnel]
    ENVFILE /usr/lib/hobbit/server/etc/hobbitserver.cfg
    CMD $BBHOME/ext/ssh-tunnels.sh
    LOGFILE $BBSERVERLOGS/ssh-tunnels.log
    INTERVAL 1m

Add entries for the Hobbit clients to the /etc/hobbit/bb-hosts file (use "noconn" if pings aren't allowed from server to client): 

xx.xx.xx.xx    clientname       # noconn ssh-tunnel

If the Hobbit clients' names cannot be resolved by the Hobbit server, add entries for those clients to the Hobbit server's /etc/hosts file: 

xx.xx.xx.xx    clientname

As of ssh-tunnel.sh version 0.0.3 you can specify a non-standard ssh port to connect to. Example: 

xx.xx.xx.xx    clientname       # noconn ssh-tunnel:2222

You could also create a DNS entry for the client, whichever is easier.

As the hobbit user, manually SSH to each Hobbit client at least once, and when prompted add the Hobbit client's host key to the Hobbit server's known_hosts file: 

$ ssh clientname
The authenticity of host 'clientname (xx.xx.xx.xx)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:x:xx:xx.
Are you sure you want to continue connecting (yes/no)?  yes

Restart the hobbit service on the Hobbit server: 

# service hobbit restart

Confirm functional

[edit | edit source]

Consult /var/log/hobbit/ssh-tunnels.log for any error messages, e.g. 

ssh: clientname: Name or service not known

Solution: Ensure that the Hobbit server can resolve the Hobbit client's name, e.g. add an entry for the Hobbit client to the Hobbit server's /etc/hosts file. 

Host key verification failed.

Solution: Manually SSH to the Hobbit client from the Hobbit server at least once and add the Hobbit client's host key to the Hobbit server's known_hosts file when prompted.

You can verify that the ssh tunnels have been successfully created: 

$ ps -ef | grep ssh
hobbit    2187     1  0 Aug14 ?        00:00:02 ssh -fnNR 1984:hobbit:1984 clientname
hobbit    2805     1  0 Aug14 ?        00:00:02 ssh -fnNR 1984:hobbit:1984 clientname2

You should also see a new column called "ssh-tunnel" generated in the Hobbit server's web interface.

Mobile phone clients

[edit | edit source]

If you want to check your hobbit / Xymon-Status from your mobile phone you can use the following apps:

Android

[edit | edit source]

XyMon by dot knowledge: http://www.dot-knowledge.de/Home/android/xymon-monitor (There you find a link to the Android market place)

Xydroid by Rune B. Broberg: http://mihtjel.dk/xydroid/ - a recent project of delivering Xymon statuses on Android phones.

See also

[edit | edit source]
Retrieved from "https://en.wikibooks.org/w/index.php?title=System_Monitoring_with_Xymon/Other_Docs/HOWTO&oldid=4410390"