SNMP Reports How-To

 

From OpenNMS

This guide asumes that you know how to play with RRDTOOL and that you are somewhat familiar with OpenNMS. All the examples presented here were tested on Redhat Linux 7.3, version 1.0 of OpenNMS with Net-SNMP 4.2.3 agents running on the managed stations.

I tried to be as accurate as possible, so read this material at your own risk.

Author:

Jose Vicente Nunez Zuleta ([e-mail])

Newbreak LLC System Administrator

Newbreak LLC

Contents

[hide]

• 1 Capture the appropiate data using SNMP (edit datacollection-config.xml)
• 2 Edit the snmp graphic properties (file snmp-graph.properties)
  • 2.1 Check your RRDTOOL files
  • 2.2 Add your graphic definitions
• 3 Add New Reports to the reports Property
• 4 Restart OpenNMS
• 5 Resources

Capture the appropiate data using SNMP (edit datacollection-config.xml)

You will need to define the OIDs you want to capture in the file datacollection-config.xml file. In our example we want to capture the following Net-SNMP OIDs (memory, host uptime and interface traffic):

 <!-- Net-SNMP memory stats, [e-mail] -->  <group name="ucd-memory" ifType="ignore"> <!-- Total Swap Size configured for the host. --> <mibObj oid=".1.3.6.1.4.1.2021.4.3" instance="0" alias="memTotalSwap" type="integer" />    <!-- Available Swap Space on the host. -->  <mibObj oid=".1.3.6.1.4.1.2021.4.4" instance="0" alias="memAvailSwap" type="integer" />    <!-- Total Real/Physical Memory Size on the host. --> <mibObj oid=".1.3.6.1.4.1.2021.4.5" instance="0" alias="memTotalReal" type="integer" />   <!-- Available Real/Physical Memory Space on the host. --> <mibObj oid=".1.3.6.1.4.1.2021.4.6" instance="0" alias="memAvailReal" type="integer" />   <!-- Error flag.  1 indicates very little swap space left --> <mibObj oid=".1.3.6.1.4.1.2021.4.100" instance="0" alias="memSwapError" type="integer" /> </group> <!-- Net-SNMP system stats, [e-mail] --> <group name="ucd-systemStats" ifType="ignore"> <!-- Amount of memory swapped to disk (kB/s). --> <mibObj oid=".1.3.6.1.4.1.2021.11.4" instance="0" alias="ssSwapOut" type="integer" />   <!-- percentages of user CPU time. --> <mibObj oid=".1.3.6.1.4.1.2021.11.9" instance="0" alias="ssCpuUser" type="integer" />   <!-- percentages of user CPU system. --> <mibObj oid=".1.3.6.1.4.1.2021.11.10" instance="0" alias="ssCpuSystem" type="integer" /  > <!-- percentages of user CPU idle. --> <mibObj oid=".1.3.6.1.4.1.2021.11.11" instance="0" alias="ssCpuIdle" type="integer" />  </group> <group  name = "mib2-interfaces-net-snmp" ifType = "all"> <mibObj oid=".1.3.6.1.2.1.2.2.1.10" instance="ifIndex" alias="ifInOctets"    type="counter"/ > <mibObj oid=".1.3.6.1.2.1.2.2.1.11" instance="ifIndex" alias="ifInUcastPkts" type="counter"/ > <mibObj oid=".1.3.6.1.2.1.2.2.1.12" instance="ifIndex" alias="ifInNUcastPkts" type="counter"/> <mibObj oid=".1.3.6.1.2.1.2.2.1.14" instance="ifIndex" alias="ifInErrors"    type="counter"/> <mibObj oid=".1.3.6.1.2.1.2.2.1.16" instance="ifIndex" alias="ifOutOctets"   type="counter"/> <mibObj oid=".1.3.6.1.2.1.2.2.1.19" instance="ifIndex" alias="ifOutDiscards" type="counter"/> <mibObj oid=".1.3.6.1.2.1.2.2.1.20" instance="ifIndex" alias="ifOutErrors"   type="counter"/> </group>

Then make sure your OIDs make it into the "systemDef" tag:

 <systemDef name = "Net-SNMP"> <!-- <sysoidMask>.1.3.6.1.4.1.2021.250.</sysoidMask> --> <sysoidMask>.1.3.6.1.4.1.2021.</sysoidMask> <collect> <includeGroup>mib2-interfaces-net-snmp</includeGroup> <includeGroup>mib2-host-resources-storage</includeGroup>  <includeGroup>mib2-host-resources-system</includeGroup> <includeGroup>mib2-host-resources-memory</includeGroup>  <includeGroup>ucd-loadavg</includeGroup> <includeGroup>ucd-systemStats</includeGroup> <includeGroup>ucd-memory</includeGroup> </collect> </systemDef>

Please note than if a given OID cannot be retrieved (for example the SNMP agent doesn't support it) then the RRDtool file is not created (normally stored in /var/opennms/rrd/snmp/)

Additional notes
Make sure <sysoidMask> matches your systems oid or else it will not find it.

$ snmpget -v2c -On -c <community> <host> sysObjectID.0.1.3.6.1.2.1.1.2.0 = OID: .1.3.6.0.0.0.0.0.0.0

would give you:

<sysoidMask>.1.3.6.0.0.0.0.0.0.0.</sysoidMask>

Edit the snmp graphic properties (file snmp-graph.properties)

Check your RRDTOOL files

You need to define your custom graphics as follow:

name

choose the name you will use for the "reports" property

columns

which rrdtool files you will use in your graphic

type

use 'interface' if the stat is related with each network interface, use 'node' if affects the whole node.

NOTE: Beginning with version 1.3.2 the types are interfaceSnmp and nodeSnmp instead of interface and node.

externalValues

database values, currently only ifSpeed

command

This is the command line you pass to rrdtool. Actualy rrdtool is called by a servlet, so you can test your graphics before adding them to this file.

One way to know wich datasources exist on your rddtool file (used in the DEF part of the rrdtool command line) is to get the information from the rrdtool file like this:

[root@lnxdev0001 16]# rrdtool info hrSystemUptime.rrdfilename = "hrSystemUptime.rrd"rrd_version = "0001"step = 300last_update = 1026942859ds[hrSystemUptime].type = "GAUGE"ds[hrSystemUptime].minimal_heartbeat = 600ds[hrSystemUptime].min = NaNds[hrSystemUptime].max = NaNds[hrSystemUptime].last_ds = "UNKN"ds[hrSystemUptime].value = 2.0296406018e+10ds[hrSystemUptime].unknown_sec = 0rra[0].cf = "AVERAGE"rra[0].rows = 8928rra[0].pdp_per_row = 1rra[0].xff = 5.0000000000e-01rra[0].cdp_prep[0].value = NaNrra[0].cdp_prep[0].unknown_datapoints = 0rra[1].cf = "AVERAGE"rra[1].rows = 8784rra[1].pdp_per_row = 12rra[1].xff = 5.0000000000e-01rra[1].cdp_prep[0].value = 7.8203629928e+08rra[1].cdp_prep[0].unknown_datapoints = 0rra[2].cf = "MIN"rra[2].rows = 8784rra[2].pdp_per_row = 12rra[2].xff = 5.0000000000e-01rra[2].cdp_prep[0].value = 7.8068611333e+07rra[2].cdp_prep[0].unknown_datapoints = 0rra[3].cf = "MAX"rra[3].rows = 8784rra[3].pdp_per_row = 12rra[3].xff = 5.0000000000e-01rra[3].cdp_prep[0].value = 7.8338616000e+07rra[3].cdp_prep[0].unknown_datapoints = 0

As you can see, the ds is 'hrSystemUptime'

If for some reason one of the rrdtool files that are part of a graphic definition is missing, then the graphic doesn't show up at all in the web console.

Remember than you can always check the contents of a rrdtool file typing:

rrdtool dump hrSystemUptime.rrd (shows a huge amount of data).

Add your graphic definitions

You can always check if the graphics you want to plot are accurate by running the rrdtool command line; Here is an example to graph the host uptime by hand:

 rrdtool graph uptime.png --title "Host uptime" --vertical-label / Days "DEF:timeticks=hrSystemUptime.rrd:hrSystemUptime:AVERAGE" / "CDEF:days=timeticks,8640000,/" AREA:days#FF0000:"Days" / GPRINT:days:AVERAGE:"Avg  //: %8.1lf %s" GPRINT:days:MIN:"Min  //: %8.1lf %s" / GPRINT:days:MAX:"Max  //: %8.1lf %s"

As you can see in the next lines, you will "copy and paste" part of this line in the graphic definition.

Here is the graphic configuration to caputure the host uptime, memory ussage and link accuracy, file snmp-graph.properties:

#### Newbreak LLC Custom reports. Jose Vicente Nunez Zuleta ([e-mail]) ###### Be very carefull with trailing spaces, otherwise you can get problems with the graphics!!!# Get the Uptime if the host uses Net-SNMP ([e-mail]). To get the graphic manually:report.netsnmp.uptime.name=Uptimereport.netsnmp.uptime.columns=hrSystemUptimereport.netsnmp.uptime.type=nodereport.netsnmp.uptime.command=--title "Host uptime" /  --vertical-label Days /  DEF:timeticks={rrd1}:hrSystemUptime:AVERAGE /  CDEF:days=timeticks,8640000,/ /  AREA:days#FF0000:"Days" /  GPRINT:days:AVERAGE:"Avg  //: %8.1lf %s" /  GPRINT:days:MIN:"Min  //: %8.1lf %s" /  GPRINT:days:MAX:"Max  //: %8.1lf %s" /# Get the Memory statistics if the host uses Net-SNMP ([e-mail]).report.netsnmp.memory.name=Memoryreport.netsnmp.memory.columns=memTotalSwap,memAvailSwap,memTotalReal,memAvailRealreport.netsnmp.memory.type=nodereport.netsnmp.memory.command=--title "Host memory usage" /  --vertical-label bytes /  DEF:mtotalSwap={rrd1}:memTotalSwap:AVERAGE /  DEF:mavailSwap={rrd2}:memAvailSwap:AVERAGE /  DEF:mtotalReal={rrd3}:memTotalReal:AVERAGE /  DEF:mavailReal={rrd4}:memAvailReal:AVERAGE /  CDEF:totalSwap=mtotalSwap,1024,* /  CDEF:availSwap=mavailSwap,1024,* /  CDEF:totalReal=mtotalReal,1024,* /  CDEF:availReal=mavailReal,1024,* /  LINE3:totalSwap#FF0000:"Total swap" /  LINE1:availSwap#00FF00:"Available swap" /  LINE3:totalReal#0000FF:"Total real" /  LINE1:availReal#000000:"Available real" /  GPRINT:totalSwap:AVERAGE:"Avg  //: %8.1lf %s" /  GPRINT:totalSwap:MIN:"Min  //: %8.1lf %s" /  GPRINT:totalSwap:MAX:"Max  //: %8.1lf %s" /  GPRINT:availSwap:AVERAGE:"Avg  //: %8.1lf %s" /  GPRINT:availSwap:MIN:"Min  //: %8.1lf %s" /  GPRINT:availSwap:MAX:"Max  //: %8.1lf %s" /  GPRINT:totalReal:AVERAGE:"Avg  //: %8.1lf %s" /  GPRINT:totalReal:MIN:"Min  //: %8.1lf %s" /  GPRINT:totalReal:MAX:"Max  //: %8.1lf %s" /  GPRINT:availReal:AVERAGE:"Avg  //: %8.1lf %s" /  GPRINT:availReal:MIN:"Min  //: %8.1lf %s" /  GPRINT:availReal:MAX:"Max  //: %8.1lf %s" /# Calculate the network accuracy using SNMP ([e-mail]).# Check the following man pages for more info on rrdtool graph:# - rrdgraph_graph# - rrdgraph# - rrdgraph_examples# Also if you forgot anbout the RPN notation (like me :) ) then go to:# - http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/tutorial/rpntutorial.html## Use CDEF to calculate this expression:# accuracy = 100 - ( (DifInErr*100) / (DifInUcast + DifInNUcast) )# In my case, Net-SNMP doesn't show up the ifInNUcastPkts OID on Linux (Solaris works fine)# but the ammount of traffic is very low, so even taking out that value the estimate is good.report.netsnmp.accuracy.name=Accuracy#report.netsnmp.accuracy.columns=ifInErrors,ifInUcastPkts,ifInNUcastPktsreport.netsnmp.accuracy.columns=ifInErrors,ifInUcastPktsreport.netsnmp.accuracy.type=interfacereport.netsnmp.accuracy.command=--title "Link accuracy" /  DEF:error={rrd1}:ifInErrors:AVERAGE /  DEF:ucast={rrd2}:ifInUcastPkts:AVERAGE /  CDEF:accuracy=100,error,100,*,ucast,/,- /  LINE2:accuracy#FF0000:"% Accuracy" /  GPRINT:accuracy:AVERAGE:"Avg  //: %8.1lf %s" /  GPRINT:accuracy:MIN:"Min  //: %8.1lf %s" /  GPRINT:accuracy:MAX:"Max  //: %8.1lf %s" /# If you are sure than all your machines have information about multicast traffic, then:# DEF:nucast={rrd3}:ifInNUcastPkts:AVERAGE /# CDEF:accuracy=100,error,100,*,ucast,nucast,+,/,- /

Add New Reports to the reports Property

Finally, don't forget to add the reports into the value of the reports property.

#report keys, list ALL prefab reports here!reports=traffic, octets, errors, discards, avgbusy5, freemem, /  bufferfails, kerneltasks, kernelmem, /  cpuPercentBusy, /  novell.numberOfNLMsLoaded, novell.openFiles, novell.licensedConnections, /  memory, /  novell.codeDataMemory, novell.cacheBuffers, /  novell.diskSpaceSys, novell.diskSpaceVol2, /  winnt2k.diskSpaceC, winnt2k.diskSpaceD, /  checkpoint.pktsAccepted, checkpoint.pktsRejected, /  checkpoint.pktsDropped, checkpoint.pktsLogged, /  loadavg, netsnmp.uptime, netsnmp.memory, netsnmp.accuracy /

Our reports are netsnmp.uptime, netsnmp.memory, netsnmp.accuracy on the last line. If you miss this step, you will get web exceptions when attempting to run performance reports. The exceptions will look something like the following:

Missing ParameterThe request you made was incomplete. It was missing the  report parameter.The following parameters are required:reportnode

Restart OpenNMS

/etc/init.d/opennms restart/etc/init.d/tomcat4 restart

Probably you will have to wait a couple of minutes before you get useful data to display.

Resources

Tutorial on how to use the RPN notation:

 http://oss.oetiker.ch/rrdtool/tut/rpntutorial.en.html

Check the following man pages for more info on rrdtool graph:

 rrdgraph_graph rrdgraph rrdgraph_examples