The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
IO-Iron-0.11
River stage one • 2 direct dependents • 2 total dependents
++
/ IO::Iron

NAME

IO::Iron - Client Libraries to Iron services IronCache, IronMQ and IronWorker.

STATUS

Iron.io libraries are currently being developed so changes in the API are possible.

SYNOPSIS

    use IO::Iron;
    use IO::Iron qw{ironcache ironmq ironworker};
    use IO::Iron ':all';

    my $iron_mq_client = ironmq();
    my @iron_mq_queues = $iron_mq_client->get_queues();

    my $iron_cache_client = ironcache( config => 'iron_cache.json' );
    my @iron_caches = $iron_cache_client->get_caches();

    my $iron_worker_client = ironworker( config => 'iron_worker.json' );
    my @iron_codes = $iron_worker_client->list_code_packages();

REQUIREMENTS

The IO::Iron::* packages require the following packages (in addition to several Perl core packages):

Carp::Assert, v. 0.20
Carp::Assert::More, v. 1.12
Data::UUID', v. 1.219,
Exception::Class, v. 1.37
File::HomeDir, v. 1.00,
File::Slurp, v. 9999.19
JSON, v. 2.53
Log::Any, v. 0.15
MIME::Base64', v. 3.13
REST::Client, v. 88
Try::Tiny, v. 0.18
URI::Escape, v. 3.31
Params::Validate, v. 1.08

IO::Iron also requires an IronIO account. Three configuration items must be set (others available) before using the functions: project_id, token and host. These can be set in a json file, as environmental variables or as parameters when creating the object.

project_id, the identification string, from IronIO.
token, an OAuth authentication token string, from IronIO.
host, the cloud in which you want to operate, e.g. 'cache-aws-us-east-1' for AWS (Amazon).

DESCRIPTION

IronCache, IronMQ and IronWorker are cloud based services accessible via a REST API. CPAN Distribution IO::Iron contains Perl clients for accessing them.

[See http://www.iron.io/]

Please see the individual clients for further documentation and usage.

Clients:

IO::Iron::IronCache::Client
IO::Iron::IronMQ::Client
IO::Iron::IronWorker::Client

IO-Iron code is available at Github: IO-Iron for download with Git: https://github.com/mikkoi/io-iron.git.

IO::Iron

Package IO::Iron is only a "convenience" module for quick startup. The three functions provided are ironcache, ironmq and ironworker.

The following parameters can be given to each of them as hash item type parameters. See section SYNOPSIS for an example.

project_id, The ID of the project to use for requests.
token, The OAuth token that is used to authenticate requests.
host, The domain name the API can be located at. E.g. 'mq-aws-us-east-1.iron.io/1'.
protocol, The protocol that will be used to communicate with the API. Defaults to "https".
port, The port to connect to the API through. Defaults to 443.
api_version, The version of the API to connect through. Defaults to the version supported by the client.
host_path_prefix, Path prefix to the RESTful url. Defaults to '/1'. Used with non-standard clouds/emergency service back up addresses.
timeout, REST client timeout (for REST calls accessing IronMQ.)
config, Config filename with path if required.

You can also give the parameters in the config file .iron.json or iron.json (in local directory) or as environmental variables. Please read Configuring the Official Client Libraries for further details.

Client Documentation

Please read individual client's documentation for using them.

Exceptions

A REST call to Iron service may fail for several reason. All failures generate an exception using the Exception::Class package. Class IronHTTPCallException contains the field status_code, response_message and error. Error is formatted as such: IronHTTPCallException: status_code=<HTTP status code> response_message=<response_message>.

    use Try::Tiny;
    use Scalar::Util qw{blessed};
    try {
        my $asked_iron_cache_01 = $iron_cache_client->get_cache('name' => 'unique_cache_name_01');
    }
    catch {
        die $_ unless blessed $_ && $_->can('rethrow');
        if ( $_->isa('IronHTTPCallException') ) {
            if ($_->status_code == 404) {
                print "Bad things! Can not just find the catch in this!\n";
            }
        }
        else {
            $_->rethrow; # Push the error upwards.
        }
    };

When using policies (see next chapter) also exceptions NoIronPolicyException and CharacterGroupNotDefinedIronPolicyException can be met.

Policies

Policies is a way to limit the names of message queues, code packages, caches and items (item keys) to a predefined group of possible strings. This can limit the chances for typos and enforce an enterprise policy. The policies are loaded from a JSON file which is specified either when creating a IO::Iron::Iron*::Client object, or in the config file .iron.json (or equivalent).

Policies in Config file

Add the item policies to the config file. The value of the item is the filename of the policies file.

Example config file:

    {
        "project_id":"51bdf5fb2267d84ced002c99",
        "token":"-Q9OEHZPhdZtd0KHBzzdUJIqV_E",
        "host":"cache-aws-us-east-1.iron.io",
        "policies":"iron_policies.json"
    }

Policies file specified when creating the client

    my $policies_filename = '/etc/ironmq/global_policies.json';
    my $client = IO::Iron::IronCache::Client->new('policies' => $policies_filename);

Examples of Policies File and Explanation of Configuration

The 'default' policies JSON file:

    (
    'definition' => {
        'character_group' => {
        },
        'no_limitation' => 1, # There is an unlimited number of alternatives.
    },
    'queue' => { 'name' => [ '[:alnum:]{1,}' ], },
    'cache' => {
        'name' => [ '[:alnum:]{1,}' ],
        'item_key' => [ '[:alnum:]{1,}' ]
        },
    'worker' => { 'name' => [ '[:alnum:]{1,}' ], },
    );

The above file would set an open policy for IronMQ, IronCache and IronWorker alike. The file is divided into four parts: definition for defining meta options, and queue|cache|worker parts for defining the changing strings (queue|cache|worker names and item keys). The character group alnum covers all ascii alphabetic characters (both lower and upper case) and digits (0-9).

N.B. The option definition:no_limitation controls the open/closed policy. If definition:no_limitation is set (1=set), the policy control is turned off.

An example of policies file

    {
        "__comment1":"Use normal regexp. [:digit:] = number:0-9, [:alpha:] = alphabetic character, [:alnum:] = character or number.",
        "__comment2":"Do not use end/begin limitators '^' and '\$'. They are added automatically.",
        "__comment3":"Note that character groups are closed inside '[::]', not '[[:]]' as normal POSIX groups.",
        'definition' => {
            'character_group' => {
                "[:lim_uchar:]":"ABC",
                "[:low_digit:]":"0123"
            },
        },
        "cache":{
            "name":[
                "cache_01_main",
                "cache_[:alpha:]{1}[:digit:]{2}"
            ],
            "item_key":[
                "item.01_[:digit:]{2}",
                "item.02_[:lim_uchar:]{1,2}"
            ]
        }
    }

This policies file sets policies for cache names and item keys. Both have two templates. Template "cache_01_main" is without wildcards: the template list can also only contain predefined names or keys. Sometimes this could be exactly the wanted behaviour, especially in regard to cache and message queue names.

Items beginning with '__' are considered comments. Comments can not be inserted into lists, such as character_group.

The definition part contains the list character_group for user-defined groups. The following groups are predefined:

[:alpha:], ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
[:alnum:], ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
[:digit:], 0123456789
[:lower:], abcdefghijklmnopqrstuvwxyz
[:upper:], ABCDEFGHIJKLMNOPQRSTUVWXYZ
[:word:], ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_

All lower ASCII (7-bit) characters are allowed in names and in character groups, except for the reserved characters (RFC 3986) !$&'()*+,;=:/?#[]@.

A character group definition is closed inside characters '[::]', not '[[:]]' as normal POSIX groups. Only the equivalents of the POSIX groups mentioned above can be used; e.g. POSIX group [[:graph:]] is not available.

When using the character groups in a name or key, only two markings are allowed: [:group:]{n} and [:group:]{n,n}, where n is an integer. This limitation (not being able to use any regular expression) is due to the double functionality of the policy: a) it acts as a filter when creating and naming new message queues, code packages, caches and cache items; 2) it can be used to list all possible names, for example when querying for cache items.

TESTING

Subdirectory integ_t contains "integration" tests which require an active Iron.io account and Internet connection. To run the tests, create first three config files in the main directory: iron_cache.json, iron_mq.json, iron_worker.json. Set at least the following attributes: project_id, token and host.

Execute prove (prove), e.g. prove -lv -Iinteg_t integ_t\Iron\iron_all.t.

FUNCTIONS

ironcache

Create an IronCache client object and return it to user.

ironmq

Create an IronMQ client object and return it to user.

ironworker

Create an IronWorker client object and return it to user.

AUTHOR

Mikko Koivunalho, <mikko.koivunalho at iki.fi>

BUGS

Please report any bugs or feature requests to bug-io-iron at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=IO-Iron. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc IO::Iron

You can also look for information at:

ACKNOWLEDGMENTS

Cool idea, "message queue in the cloud": http://www.iron.io/. And well implemented, too, with webhooks for several functions!

TODO

  • Fix the $self->{'caches'} to a hash instead of list.

  • Implement new features in http://blog.iron.io/2014/05/ironmq-long-polling.html

    - Long Polling
    - Get-Delete as One Operation

  • The IronMQ client needs to control the queues, perhaps using semafores.

  • A buffer mechanism to keep the messages while the IronMQ REST service is unavailable. IO::Iron::IronMQ::ASyncPush?

  • Implement push queues.

  • Mock IronMQ/IronCache for testing.

  • Handle message size (total), delay, timeout and expiration min-max values.

    • Message Var Default Maximum Notes

    • Message Size -- 64KB Includes the entire request (delay, timeout, expiration).

    • Delay 0sec 604,800sec Message is made available on queue after the delay expires.

    • Timeout 60sec 86,400sec Message goes back on queue after timeout unless deleted.

    • Expiration 604,800sec 2,592,000sec Equates to 7 days and 30 days, respectively.

    • Messages per Get 1 100 One or more messages can be handled at a time.

  • Option to delete queue when IO::Iron::IronMQ::Queue object goes to garbage collection?

  • Verify the client is connected when created (by calling queues?)

  • Rethink the using of REST:Client. Since message queues often involve a lot of traffic but always to the same address, we need to optimize REST:Client usage.

  • Carp::Assert, Carp::Assert::More, other checking deactivation in production?

LICENSE AND COPYRIGHT

Copyright 2013 Mikko Koivunalho.

This program is free software; you can redistribute it and/or modify it under the terms of the the Artistic License (2.0). You may obtain a copy of the full license at:

http://www.perlfoundation.org/artistic_license_2_0

Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license.

If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license.

This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder.

This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed.

Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.