RT Nagios integration

Nagios and Request Tracker Integration

Creating Tickets

Introduction

Goals

Notes

Exercises

To configure RT and Nagios so that alerts from Nagios automatically create tickets requires a few steps:

These next two items should already be done in RT if you have finished the RT exercises.

Exercises

Log in to your virtual machine (hostX-campusY) as the “sysadm” user.

Configure a Contact in Nagios

Become root on your host:

$ sudo -s

Edit the file /etc/nagios4/objects/contacts.cfg

# editor /etc/nagios4/objects/contacts.cfg

In this file we will first add a new contact name under the default root contact entry.

WARNING: DO NOT remove the root contact entry.

REALLY - READ THIS! DO NOT remove the “nagiosadmin” contact entry.

ADD the entry below just AFTER the ROOT contact entry

The new contact should look like this (UNDERNEATH the nagiosadmin contact entry):

define contact {
        contact_name                    net
        alias                           RT Alert Queue
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options    c 
        host_notification_options       d 
        service_notification_commands   notify-service-by-email
        host_notification_commands      notify-host-by-email
        email                           net@srv1-campusY.ws.nsrc.org
}

Be sure that you replace “campusY” with the number of your group. So, if you are in group 1, then you would use “net@srv1-campus1.ws.nsrc.org”.

At the end of the file add the following entry: (Note: do not delete or update other entries.)

DO THIS AT THE END OF THE FILE

define contactgroup {
        contactgroup_name       tickets
        alias                   email to ticket system for RT
        members                 net
}

Save and exit from the file.

Notes

Choose a Service to Monitor that Creates Tickets in RT

To send email to generate tickets in RT if SSH goes down on a box you would edit the SSH service check:

# editor /etc/nagios4/conf.d/services.cfg

Find the service with the hostgroup_name of “ssh-servers” and add the “contact_groups” entry at the end of the definition. When you are done your definition should look like this. Also change “max_check_attempts” from “10” to “4”.

define service {
        hostgroup_name                  ssh-servers
        service_description             SSH
        check_command                   check_ssh
        use                             generic-service
        # ssh occasionally flaps. Let's be sure it's down.
        retry_interval                  2
        max_check_attempts              4
        notification_interval           0 ; set > 0 if you want to be renotified
        contact_groups                  tickets,admins
}

Change max_check_attempts to “4” so that Nagios send an E-mail to RT much more quickly. If we leave max_check_attempts at “10” then the process is slower.

Save and exit from the file.

Notes

Include noc as members of ssh-server

# editor /etc/nagios4/conf.d/hostgroups.cfg

Find the group with the hostgroup_name of “ssh-servers” and add noc to the “members” entry at the end of the definition. When you are done your definition should looke like this:

define hostgroup {
  hostgroup_name  ssh-servers
  alias           SSH Servers
  members         srv1-campusY, host1-campusY, host2-campusY, host3-campusY, \
                  host4-campusY, host5-campusY, host6-campusY, transit1-nren, \
                  bdr1-campusY, core1-campusY, dist1-b1-campusY, dist1-b2-campusY, \
                  noc
}

Now save and exit from the file.

To create a "NOC entry add this to the file pcs.cfg at the bottom of the file:

define host {
    use         linux-server
    host_name   noc
    alias       Main NOC server for NMM Workshop
    address     noc.ws.nsrc.org
}

Save and exit from the file.

Now verify that your Nagios configuration looks good:

# nagios4 -v /etc/nagios4/nagios.cfg

If there are no errors, then restart your nagios instance:

# systemctl restart nagios4

If SSH goes down on any of the devices you are monitoring Nagios should generate a new ticket in Request Tracker. We will stop the SSH service on the classroom NOC server.

Bring down the SSH service on NOC or another server (optional)

Right now you are monitoring noc.ws.nsrc.org for the ssh service. You can ask an instructor to take down the service.

But you can also work within your groups to make changes to the ssh servers in your campus to generate RT tickets. On any server which is monitored by your nagios server, for example hostX-campusY, you can do the following:

$ ssh sysadm@hostX-campusY.ws.nsrc.org
$ sudo vi /etc/ssh/sshd_config

Change the line:

#Port 22

to:

#Port 22
Port 2222

Then restart the ssh service:

$ sudo systemctl restart ssh

If you log out of the server you can log back in via ssh by doing:

$ ssh -p 2222 sysadm@hostX-campus.Y.ws.nsrc.org

and to put it back you can remove the line

Port 2222

from /etc/ssh/sshd_config and type.

$ sudo systemctl restart ssh

If you are using the workshop web interface it is possible to reconnect to hostX-campusY using the “Console” option to the srv1 server, and once on there use incus shell hostX-campusY to get a shell on any of the hosts.

See Nagios Tickets in RT