The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
RRD-Editor-0.15
River stage zero No dependents
1 ++
/ RRD::Editor

NAME

RRD::Editor - Portable, standalone (no need for RRDs.pm) tool to create and edit RRD files.

SYNOPSIS

 use strict;
 use RRD::Editor ();
 
 # Create a new object
 my $rrd = RRD::Editor->new();
  
 # Create a new RRD with 3 data sources called bytesIn, bytesOut and 
 # faultsPerSec and one RRA which stores 1 day worth of data at 5 minute 
 # intervals (288 data points). The argument format is the same as that used 
 # by 'rrdtool create', see L<http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html>
 $rrd->create("DS:bytesIn:GAUGE:600:U:U DS:bytesOut:GAUGE:600:U:U DS:faultsPerSec:COUNTER:600:U:U RRA:AVERAGE:0.5:1:288")

 # Save RRD to a file
 $rrd->save("myfile.rrd");
 # The file format to use can also be optionally specified:
 # $rrd->save("myfile.rrd","native-double");   # default; non-portable format used by RRDTOOL
 # $rrd->save("myfile.rrd","portable-double"); # portable, data stored in double-precision
 # $rrd->save("myfile.rrd","portable-single"); # portable, data stored in single-precision

 # Load RRD from a file.  Automagically figures out the file format 
 # (native-double, portable-double etc) 
 $rrd->open("myfile.rrd");
 
 # Add new data to the RRD for the same 3 data sources bytesIn, 
 # bytesOut and faultsPerSec.  The argument format is the same as that used by 
 # 'rrdtool update', see L<http://oss.oetiker.ch/rrdtool/doc/rrdupdate.en.html>
 $rrd->update("N:10039:389:0.4");
  
 # Show information about an RRD.  Output generated is similar to 
 # 'rrdtool info'.
 print $rrd->info();
 
 # XML dump of RRD contents.  Output generated is similar to 'rrdtool dump'.
 print $rrd->dump();
 
 # Extract data measurements stored in RRAs of type AVERAGE
 # The argument format is the same as that used by 'rrdtool fetch' and 
 # the output generated is also similar, see
 # L<http://oss.oetiker.ch/rrdtool/doc/rrdfetch.en.html>
 print $rrd->fetch("AVERAGE");
 
 # Get the time when the RRD was last updated (as a unix timestamp)
 printf "RRD last updated at %d\n", $rrd->last();

 # Get the measurements added when the RRD was last updated
 print $rrd->lastupdate();
 
 # Get the min step size (or resolution) of the RRD.  This defaults to 300s unless specified
 otherwise when creating an RRD.
 print $rrd->minstep()

Edit Data Sources

 # Add a new data-source called bytes.  Argument format is the same as $rrd->create().
 $rrd->add_DS("DS:bytes:GAUGE:600:U:U");
 
 # Delete the data-source bytesIn
 $rrd->delete_DS("bytesIn");
 
 # Get a list of the data-sources names
 print $rrd->DS_names();
 
 # Change the name of data-source bytes to be bytes_new
 $rrd->rename_DS("bytes", "bytes_new")
 
 # Get the heartbeat value for data-source bytesOut (the max number of seconds that
 # may elapse between data measurements)
 printf "Heartbeat for DS bytesOut = %d\n", $rrd->DS_heartbeat("bytesOut");

 # Set the heartbeat value for data-source bytesOut to be 1200 secs
 $rrd->set_DS_heartbeat("bytesOut",1200);
 
 # Get the type (COUNTER, GAUGE etc) of data-source bytesOut
 printf "Type of DS bytesOut = %s\n", $rrd->DS_type("bytesOut");
 
 # Set the type of data-source bytesOut to be COUNTER
 $rrd->set_DS_type("bytesOut", "COUNTER");
 
 # Get the minimum value allowed for measurements from data-source bytesOut
 printf "Min value of DS bytesOut = %s\n", $rrd->DS_min("bytesOut");

 # Set the minimum value allowed for measurements from data-source bytesOut to be 0
 $rrd->set_DS_min("bytesOut",0);
 
 # Get the maximum value allowed for measurements from data-source bytesOut
 printf "Max value of DS bytesOut = %s\n", $rrd->DS_max("bytesOut");
 
 # Set the maximum value allowed for measurements from data-source bytesOut to be 100
 $rrd->set_DS_max("bytesOut",100);

Edit RRAs

 # Add a new RRA which stores 1 weeks worth of data (336 data points) at 30 minute  
 # intervals (30 mins = 6 x 5 mins)
 $rrd->add_RRA("RRA:AVERAGE:0.5:6:336");

 # RRAs are identified by an index in range 0 .. $rrd->num_RRAs().  The index 
 # of an RRD can also be found using $rrd->info() or $rrd->dump()
 my $rra_idx=1; 
 
 # Delete an existing RRA with index $rra_idx.  
 $rrd->delete_RRA($rra_idx);
 
 # Get the number of rows/data points stored in the RRA with index $rra_idx
 $rra_idx=0; 
 printf "number of rows of RRA %d = %d\n", $rra_idx, $rrd->RRA_numrows($rra_idx);
 
 # Change the number of rows/data points stored in the RRA with index 
 # $rra_idx to be 600.
 $rra->resize_RRA($rra_idx, 600);
 
  # Get the value of bytesIn stored at the 10th row/data-point in the 
 # RRA with index $rra_idx.
 printf "Value of data-source bytesIn at row 10 in RRA %d = %d", $rra_idx, $rra->RRA_el($rra_idx, "bytesIn", 10);
 
  # Set the value of bytesIn at the 10th row/data-point to be 100
 $rra->set_RRA_el($rra_idx, "bytesIn", 10, 100);  
 
 # Get the xff value for the RRA with index $rra_idx
 printf "Xff value of RRA %d = %d\n", $rra_idx, $rra->RRA_xff($rra_idx);

 # Set the xff value to 0.75 for the RRA with index $rra_idx
 $rra->RRA_xff($rra_idx,0.75);
 
 # Get the type (AVERAGE, LAST etc) of the RRA with index $rra_idx
 print $rrd->RRA_type($rra_idx);
 
 # Get the step (in seconds) of the RRA with index $rra_idx
 print $rrd->RRA_step($rra_idx);

DESCRIPTION

RRD:Editor implements most of the functionality of RRDTOOL, apart from graphing, plus adds some new editing and portability features. It aims to be portable and self-contained (no need for RRDs.pm).

RRD::Editor provides the ability to add/delete DSs and RRAs and to get/set most of the parameters in DSs and RRAs (renaming, resizing etc). It also allows the data values stored in each RRA to be inspected and changed individually. That is, it provides almost complete control over the contents of an RRD.

The RRD files created by RRDTOOL use a binary format (let's call it native-double) that is not portable across platforms. In addition to this file format, RRD:Editor provides two new portable file formats (portable-double and portable-single) that allow the exchange of files. RRD::Editor can freely convert RRD files between these three formats (native-double,portable-double and portable-single).

Notes:

  • times must all be specified as unix timestamps (i.e. -1d, -1w etc don't work, and there is no @ option in rrdupdate).

  • there is full support for COUNTER, GAUGE, DERIVE and ABSOLUTE data-source types but the COMPUTE type is only partially supported.

  • there is full support for AVERAGE, MIN, MAX, LAST RRA types but the HWPREDICT, MHWPREDICT, SEASONAL etc types are only partially supported).

METHODS

new

 my $rrd=new RRD:Editor->new();

Creates a new RRD::Editor object

create

 $rrd->create($args);

The method will create a new RRD with the data-sources and RRAs specified by $args. $args is a string that contains the same sort of command line arguments that would be passed to rrdtool create. The format for $args is:

[--start|-b start time] [--step|-s step] [--format|-f encoding] [DS:ds-name:DST:heartbeat:min:max] [RRA:CF:xff:steps:rows]

where DST may be one of GAUGE, COUNTER, DERIVE, ABSOLUTE and CF may be one of AVERAGE, MIN, MAX, LAST. Possible values for encoding are native-double, portable-double, portable-single. If omitted, defaults to native-double (the non-portable file format used by RRDTOOL). See http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html for further information.

open

 $rrd->open($file_name);

Load the RRD in the file called $file_name. Only the file header is loaded initially, to improve efficiency, with the body of the file subsequently loaded if needed. The file format (native-double, portable-double etc) is detected automagically.

save

 $rrd->save();
 $rrd->save($file_name);
 $rrd->save($file_name, $encoding);

Save RRD to a file called $file_name with format specified by $encoding. Possible values for $encoding are "native-double", "portable-double", "portable-single".

If omitted, $encoding defaults to the format of the file specified when calling open(), or to "native-double" if the RRD has just been created using create(). native-double is the non-portable binary format used by RRDTOOL. portable-double is portable across platforms and stores data as double-precision values. portable-single is portable across platforms and stores data as single-precision values (reducing the RRD file size by approximately half). If interested in the gory details, portable-double is just the native-double format used by Intel 32-bit platforms (i.e. little-endian byte ordering, 32 bit integers, 64 bit IEEE 754 doubles, storage aligned to 32 bit boundaries) - an arbitrary choice, but not unreasonable since Intel platforms are probably the most widespread at the moment, and it is also compatible with web tools such as javascriptRRD http://javascriptrrd.sourceforge.net/.

If the RRD was opened using open(), then $file_name is optional and if omitted save() will save the RRD to the same file as it was read from.

close

 $rrd->close();

Close an RRD file accessed using open() or save(). Calling close() flushes any cached data to disk.

info

 my $info = $rrd->info();

Returns a string containing information on the DSs and RRAs in the RRD (but not showing the data values stored in the RRAs). Also shows details of the file format (native-double, portable-double etc) if the RRD was read from a file.

dump

 my $dump = $rrd->dump();
 my $dump = $rrd->dump($arg);

Returns a string containing the complete contents of the RRD (including data) in XML format. $arg is optional. Possible values are "--no-header" or "-n", which remove the XML header from the output string.

fetch

 my $vals = $rrd->fetch($args);

Returns a string containing a table of measurement data from the RRD. $args is a string that contains the same sort of command line arguments that would be passed to rrdtool fetch. The format for $args is:

 CF [--resolution|-r resolution] [--start|-s start] [--end|-e end]

where CF may be one of AVERAGE, MIN, MAX, LAST. See http://oss.oetiker.ch/rrdtool/doc/rrdfetch.en.html for further details.

update

 $rrd->update($args);

Feeds new data values into the RRD. $args is a string that contains the same sort of command line arguments that would be passed to rrdtool update. The format for $args is:

 [--template:-t ds-name[:ds-name]...] N|timestamp:value[:value...] [timestamp:value[:value...] ...]

See http://oss.oetiker.ch/rrdtool/doc/rrdupdate.en.html for further details.

Since update() is often called repeatedly, in-place updating of RRD files is used where possible for greater efficiency . To understand this, a little knowledge of the RRD file format is needed. RRD files consist of a small header containing details of the DSs and RRAs, and a large body containing the data values stored in the RRAs. Reading the body into memory is relatively costly since it is much larger than the header, and so is only done by RRD::Editor on an "as-needed" basis. So long as the body has not yet been read into memory when update() is called, update() will update the file on disk i.e. without reading in the body. In this case there is no need to call save(). If the body has been loaded into memory when update() is called, then the copy of the data stored in memory will be updated and the file on disk left untouched - a call to save() is then needed to freshen the file stored on disk. Seems complicated, but its actually ok in practice. If all you want to do is efficiently update a file, just use the following formula:

 $rrd->open($file_name);
 $rrd->update($args);
 $rrd->close();

and that's it. If you want to do more, then be sure to call save() when you're done.

last

 my $unixtime = $rrd->last();

Returns the time when the data stored in the RRD was last updated. The time is returned as a unix timestamp. This value should not be confused with the last modified time of the RRD file.

set_last

 $rrd->set_last($unixtime);

Set the last update time to equal $unixtime. WARNING: Rarely needed, use with caution !

lastupdate

 my @vals=$rrd->lastupdate();

Return a list containing the data-source values inserted at the most recent update to the RRD

minstep

 my $minstep = $rrd->minstep();

Returns the minimum step size (in seconds) used to store data values in the RRD. RRA data intervals must be integer multiples of this step size. The min step size defaults to 300s when creating an RRD (where it is referred to as the "resolution"). NB: Changing the min step size is hard as it would require resampling all of the stored data, so we leave this "to do".

add_DS

 $rrd->add_DS($arg);

Add a new data-source to the RRD. Only one data-source can be added at a time. Details of the data-source to be added are specified by the string $arg. The format of $arg is:

 [DS:ds-name:DST:heartbeat:min:max]

where DST may be one of GAUGE, COUNTER, DERIVE, ABSOLUTE i.e. the same format as used for create().

delete_DS

 $rrd->delete_DS($ds-name);

Delete the data-source with name $ds-name from the RRD. WARNING: This will irreversibly delete all of the data stored for the data-source.

DS_names

 my @ds-names = $rrd->DS_names();

Returns a list containing the names of the data-sources in the RRD.

rename_DS

 $rrd->rename_DS($ds-name, $ds-newname);

Change the name of data-source $ds-name to be $ds-newname

DS_heartbeat

 my $hb= $rrd->DS_heartbeat($ds-name);

Returns the current heartbeat (in seconds) of a data-source. The heartbeat is the max number of seconds that may elapse between data measurements before declaring that data is missing.

set_DS_heartbeat

 $rrd->set_DS_heartbeat($ds-name,$hb);

Sets the heartbeat value (in seconds) of data-source $ds-name to be $hb.

DS_type

 my $type = $rrd->DS_type($ds-name);

Returns the type (GAUGE, COUNTER etc) of a data-source.

set_DS_type

 $rrd->set_DS_type($ds-name, $type);

Sets the type of data-source $ds-name to be $type.

DS_min

 my $min = $rrd->DS_min($ds-name);

Returns the minimum allowed for measurements from data-source $ds-name. Measurements below this value are set equal to $min when stored in the RRD.

set_DS_min

 $rrd->set_DS_min($ds-name, $min);

Set the minimum value for data-source $ds-name to be $min.

DS_max

 my $max = $rrd->DS_max($ds-name);

Returns the maximum allowed for measurements from data-source $ds-name. Measurements above this value are set equal to $max when stored in the RRD.

set_DS_max

 $rrd->set_DS_max($ds-name, $max);

Set the maximum value for data-source $ds-name to be $max.

add_RRA

 $rrd->add_RRA($arg);

Add a new RRA to the RRD. Only one RRA can be added at a time. Details of the RRA to be added are specified by the string $arg. The format of $arg is:

 [RRA:CF:xff:steps:rows]

where CF may be one of AVERAGE, MIN, MAX, LAST i.e. the same format as used for create().

num_RRAs

 my $num_RRAs = $rrd->num_RRAs();

Returns the number of RRAs stored in the RRD. Unfortunately, unlike data-sources, RRAs are not named and so are only identified by an index in the range 0 .. num_RRAs()-1. The index of a specific RRD can be found using info() or dump().

delete_RRA

 $rrd->delete_RRA($rra_idx);

Delete the RRA with index $rra_idx (see above discussion for how to determine the index of an RRA). WARNING: This will irreversibly delete all of the data stored in the RRA.

RRA_numrows

 my $numrows = $rrd->RRA_numrows($rra_idx);

Returns the number of rows in the RRA with index $rra_idx.

resize_RRA

 $rra->resize_RRA($rra_idx, $numrows);

Change the number of rows to be $numrows in the RRA with index $rra_idx. WARNING: If $numrows is smaller than the current row size, excess data points will be discarded.

RRA_el

 my ($t,$val) = $rra->RRA_el($rra_idx, $ds-name, $row);

Returns the timestamp and the value of data-source $ds-name stored at row $row in the RRA with index $rra_idx. $row must be in the range [0 .. RRA_numrows($rra_idx)-1]. Row 0 corresponds to the oldest data point stored and row RRA_numrows($rra_idx)-1 to the most recent data point.

set_RRA_el

 $rra->set_RRA_el($rra_idx, $ds-name, $row, $val);

Set the stored value equal to $val for data-source $ds-name stored at row $row in the RRA with index $rra_idx.

RRA_xff

 my $xff = $rra->RRA_xff($rra_idx);

Returns the xff value for the RRA with index $rra_idx. The xff value defines the proportion of an RRA data interval that may contain UNKNOWN data (i.e. missing data) and still be treated as known. For example, an xff value 0.5 in an RRA with data interval 300 seconds (5 minutes) means that if less than 150s of valid data is available since the last measurement, UNKNOWN will be stored in the RRA for the next data point.

set_RRA_xff

 $rra->RRA_xff($rra_idx,$xff);

Sets the xff value to $xff for the RRA with index $rra_idx.

RRA_step

 my $step = $rrd->RRA_step($rra_idx);

Returns the data interval (in seconds) of the RRA with index $rra_idx. NB: Changing the step size is hard as it would require resampling the data stored in the RRA, so we leave this "to do".

RRA_type

 my $type = $rrd->RRA_type($rra_idx);

Returns the type of the RRA with index $rra_idx i.e. AVERAGE, MAX, MIN, LAST etc. NB: Changing the type of an RRA is hard (impossible ?) as the stored data doesn't contain enough information to allow its type to be changed. To change type, its recommended instead to delete the RRA and add a new RRA with the desired type.

EXPORTS

You can export the following functions if you do not want to use the object orientated interface:

 create
 open
 save
 close
 update
 info
 dump
 fetch
 last 
 set_last
 lastupdate
 minstep
 add_RRA
 delete_RRA
 num_RRAs
 RRA_numrows
 resize_RRA
 RRA_type
 RRA_step
 RRA_xff
 set_RRA_xff
 add_DS
 delete_DS
 DS_names
 rename_DS
 DS_heartbeat
 set_DS_heartbeat
 DS_min
 set_DS_min
 DS_max
 set_DS_max
 DS_type
 set_DS_type

The tag all is available to easily export everything:

 use RRD::Editor qw(:all);

Portability/Compatibility with RRDTOOL

The RRD::Editor code is portable, and so long as you stick to using the portable-double and portable-single file formats the RRD files generated will also be portable. Portability issues arise when the native-double file format of RRD::Editor is used to store RRDs. This format tries to be compatible with the non-portable binary format used by RRDTOOL, which requires RRD::Editor to figure out nasty low-level details of the platform it is running on (byte ordering, byte alignment, representation used for doubles etc). To date, RRD::Editor and RRDTOOL have been confirmed compatible (i.e. they can read each others RRD files in native-double format) on the following platforms:

Intel 386 32bit, Intel 686 32bit, AMD64/Intel x86 64bit, Itanium 64bit, Alpha 64bit, MIPS 32bit, MIPSel 32 bit, MIPS 64bit, PowerPC 32bit, ARM 926EJ-S v5l (32bit, ABI/mixed-endian floats), SPARC 32bit, SPARC 64bit, SH4

Known issues:

On ARM EABI platforms RRD::Editor and RRDTOOL only partially compatible if RRDTOOL compiled with ABI support instead of EABI (RRD::Editor can read RRDTOOL files, but not vice-versa)

For more information on RRD::Editor portability testing, see http://www.glmonitoring.com/rrdeditor/. If your platform is not listed, there is a good chance things will "just work" but double checking that RRDTOOL can read the native-double format RRD files generated by RRD::Editor, and vice-versa, would be a good idea if that's important to you.

SEE ALSO

rrdtool.pl command line interface for RRD::Editor, RRD::Simple, RRDTool::OO, http://www.rrdtool.org

VERSION

Ver 0.15

AUTHOR

Doug Leith

http://www.glmonitoring.com

BUGS

Please report any bugs or feature requests to bug-rrd-db at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=RRD-Editor. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

COPYRIGHT

Copyright 2012 D.J.Leith.

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.

See http://dev.perl.org/licenses/ for more information.