Java(TM) Execution Time Measurement Library

Runtime performance monitoring made easy

RRD4j offline integration

This page briefly describes RRD4j offline support. In offline mode we will record raw performance statistics at runtime, log the collected data to a file and import these data from the file to a RRD database manually. The import will be done through a command line tool and can happen at any time (if the order is preserved).

The same command line tool can be used to create RRD graphs similar to the ones presented here.

Step 1: Prerequisites

In offline mode there are no additional JETM runtime requirements apart from minor configuration changes.

The command line tool however depends on RRD4j that can be downloaded here. Simply put jetm.jar, jetm-optional.jar and all RRD4j libraries to your classpath and execute etm.contrib.rrd.rrd4j.Rrd4jMain. RRD4j and Rrd4jMain require Java™ 5.0 or higher.

Step 2: Enable raw data logging

The RRD4j offline mode uses raw performance data that can be created by a JETM runtime on-the-fly. Therefore raw-data logging has to be enabled (which is disabled by default). The easiest way enable this feature is to add the configuration option raw-data-log to a static EtmManger or Springframework based configuration or add a Logging Aggregator to the aggregation chain.

The following example enables raw-data logging for an EtmManager based configuration and logs raw performance data to log4j using the default category etm-raw-data at level INFO. The optional filter SoapAction.+;HTTP.+ ensures that only certain measurement points will be logged.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jetm-config PUBLIC "-// void.fm //DTD JETM Config 1.2//EN"
                             "http://jetm.void.fm/dtd/jetm_config_1_2.dtd">

<jetm-config>
  <features>
    ...
    <raw-data-log type="log4j" filter-pattern="^SoapAction.+;^HTTP.+" />
  </features>
</jetm-config>

It is recommended to configure the log framework such that all raw performance data are logged to dedicated log files on a daily basis. While JETM is capable to differentiate between historical and new performance data it has to scan every line in a log file to locate new measurements. If you use one log file only you will notice significant import performance degration over time.

Step 3: Create target RRD databases

The command line tool etm.contrib.rrd.rrd4j.Rrd4jMain supports the import of raw performance data to an already existing RRD DB. Therefore it is up to you to create those files upfront. The following example creates two RRD DB files - one for Soap Actions and another one for HTTP requests - using a high resolution database. Apart from the highres database template JETM ships with templates for mediumres and lowres databases. All templates define 4 datasource collecting min, average and maximum execution times and the number of executions (called transactions!).

java etm.contrib.rrd.rrd4j.Rrd4jMain create-db -t highres -d /tmp/soap_actions.rrd
java etm.contrib.rrd.rrd4j.Rrd4jMain create-db -t highres -d /tmp/http_requests.rrd

Step 4: Import raw data to RRD DB

Once you have created your RRD DB files you are ready to import raw performance data. The following example imports a selected set of performance results from a single log file to dedicated RRD DB files. Please note that you can specifcy multiple RRD DB destinations at the same time or use the same file multiple times using distinct destinations.

java etm.contrib.rrd.rrd4j.Rrd4jMain import -s /tmp/performance.log
     -d /tmp/soap_actions.rrd!^SoapAction.+;/tmp/http_requests.rrd!^HTTP.+

You may run an import from the same file multiple times - the import tool will detect whether the given file contains newer performance data or not. However is impossible to import historical data once you have imported more recent ones. Therefore you need to ensure to preserve the logging order while importing from multiple files.

Step 5: Create performance graphs

After the import the stored data can be rendered. The following examples render maximum and average times found in our RRD DB files for given intervals (last hour and last 24 hours) and use the pre-defined render templates max-average. Apart from the max-average image template JETM ships with templates for min, max, average, min-average, tx and average-tx renderings, whereas tx means transactions.

java etm.contrib.rrd.rrd4j.Rrd4jMain create-graph -t max-average -s /tmp/soap_actions.rrd
  -d /tmp/performance-soap-actions-last-hour.png
  -p imagetitle="Soap Requests last hour"

java etm.contrib.rrd.rrd4j.Rrd4jMain create-graph -t max-average -s /tmp/soap_actions.rrd
  -d /tmp/performance-soap-actions-last-24-hours.png -i 24h
  -p imagetitle="Soap Requests last 24 hours"


java etm.contrib.rrd.rrd4j.Rrd4jMain create-graph -t max-average -s /tmp/http_requests.rrd
  -d /tmp/performance-http-requests-last-hour.png
  -p imagetitle="HTTP Requests last hour"

java etm.contrib.rrd.rrd4j.Rrd4jMain create-graph -t max-average -s /tmp/http_requests.rrd
  -d /tmp/performance-http-requests-last-24-hours.png -i 24h
  -p imagetitle="HTTP Requests last 24 hours"

Attention: The import tool will import performance data at recording time. Therefore you may not be able to see rendered performance results using the commands above (since your data may be too old for the selected intervals). The command line tool supports several ways to specify a certain monitoring interval. See table below for further options.

During the image rendering process RRD4j makes uses of AWT. In server applications it is usually a good idea to switch to awt headless mode by specifying -Djava.awt.headless=true.

Full list of Rrd4jMain options

The following table contains all supported etm.contrib.rrd.rrd4j.Rrd4jMain options. The field mandatory indicates that these options are required for all JETM pre-defined RrdDef and RrdGraphDef templates.

Mode	Option	Meaning	Mandatory
create-db	t	The DB template to use, available in classpath or file system. Should follow RrdDefTemplate format. Pre-defined templates are highres, mediumres and lowres.	Yes
	d	Database destination, use absolute path if possible.	Yes
	p	Template properties to be supplied to the template. Use key=value separated by semicolon for multiple values. E.g. -p foo=bar;yadda=123	No
	Examples: Create a high resoslution rrd database called test.rrd: Rrd4jMain create-db -t highres -d /tmp/test.rrd
import	s	The source file. Use absolute file path if possible.	Yes
	d	The parsing definitions. A single parsing definition is made of the path to the rrd db file and a Java 1.4 regular expression used to select measurement results separated by !. Multiple definitions can be separated using a semilocon. If you want to import all results use * instead of regular expression.	Yes
	p	Template properties to be supplied to the template. Use key=value separated by semicolon for multiple values. E.g. -p foo=bar;yadda=123	No
	Examples: Import all results to test.rrd from performance.log: Rrd4jMain import -s /tmp/performance.log -d /tmp/test.rrd!* Imports all SoapActions to foo.rrd and HTTP requests to bar.rrd: Rrd4jMain import -s /tmp/performance.log -d /tmp/foo.rrd!^SoapAction.+;/tmp/bar.rrd!^HTTP.+
create-graph	s	The RRD DB to read from.	Yes
	d	The name of the image. Creates PNG images by default.	Yes
	t	The graph template to use, available in classpath or file system. Should follow RrdGraphDefTemplate format. Pre-defined templates are min, max, average, min-average, max-average, tx and average-tx.	Yes
	i	The render interval in hours, days, months or years [h,d,m,y]. By default the interval is calculated from now to the past unless -o or -b is specified. Examples: 12m, 5d.	No (Last hour)
	o	The interval offset in hours, days, months or years [h,d,m,y]. Moves the interval end by the given offset to the past. Use ! to move the interval end to 12:00am. Ingored if -b is specified. Examples: 1d, 5d!, 0d!.	No
	b	The begin date of the interval in YYYY.MM.DD. The interval always starts at 12.00am and can be combined with -e or -i. Example: 2007.02.28	No
	e	The end date of the interval in YYYY.MM.DD. The interval always ends at 12.00am and requires -b. Ignored otherwhise. Example: 2007.03.01	No
	p	Template properties to be supplied to the template. Use key=value separated by semicolon for multiple values. E.g. -p foo=bar;yadda=123	No
	Examples: Renders average and maximum times for the last hour Rrd4jMain create-graph -s /tmp/test.rrd -d /tmp/test.png -t max-average Renders maximum times for the last 7 days Rrd4jMain create-graph -s /tmp/test.rrd -d /tmp/test.png -t max -i 7d Renders total transactions for the last 7 days moving the interval end to yesterday 12.00am Rrd4jMain create-graph -s /tmp/test.rrd -d /tmp/test.png -t tx -i 7d -o 0d! Renders minimum times for 30 days past 2007.01.01 Rrd4jMain create-graph -s /tmp/test.rrd -d /tmp/test.png -t min -b 2007.01.01 -i 30d Renders average times for the interval 2007.01.01-2008.01.01 Rrd4jMain create-graph -s /tmp/test.rrd -d /tmp/test.png -t average -b 2007.01.01 -e 2008.01.01 Renders average and maximum times for the last hour and sets image title Rrd4jMain create-graph -s /tmp/test.rrd -d /tmp/test.png -t max-average -p "imagetitle=MyHost performance last hour"