POE::Component::Client::MogileFS - an async MogileFS client for POE
Version 0.02
use POE qw(Component::Client::MogileFS); my $num = 500; my @tasks; foreach (1..$num) { my $key = $_ .'x'. int(rand($num)) . time(); my $data = $key x 1000; push @tasks, { method => 'store_content', domain => 'testdomain', trackers => [qw/192.168.0.31:6001/], args => [$key, 'testclass', $data], taskname => $key.':testclass:testdomain', }; } POE::Session->create( inline_states => { _start => \&start_session, storesomestuff => \&storesomestuff, debugging => \&debugging, } ); sub start_session { my ($kernel, $heap, $session) = @_[KERNEL, HEAP, SESSION]; POE::Component::Client::MogileFS->spawn( alias => 'mog', max_concurrent => 50, result => [$session, 'storesomestuff'], debug => [$session, 'debugging'], ); $kernel->post('mog', 'add_tasks', \@tasks); } my $count = 0; sub storesomestuff { my ($kernel,$result) = @_[KERNEL,ARG0]; if ($result->{status}) { $count++; print "$count RESULT ".$result->{task}->{taskname}; print " SUCCESS ".$result->{status}."\n"; } else { print "$count RESULT ".$result->{task}->{taskname}; print " FAILED\n"; $kernel->post('mog', 'add_tasks', [$result->{task}]); } $kernel->post('mog', 'shutdown') if $count == $num; } sub debugging { my $result = $_[ARG0]; print "DEBUG $result\n"; } $poe_kernel->run();
POE::Component::Client::MogileFS is a POE component that uses Wheel::Run to fork off child processes which will execute MogileFS::Client methods with your provided data asyncronously. By default it will not allow more than 10 concurrent connections, but you can adjust that as needed. This is my first go at a POE::Component so the api may change in future, and I'm really open to suggestions for improvement/features.
Can take the following arguments: alias => 'alias of mogilefs session or __PACKAGE__ by default', max_concurrent => 'max number of concurrent children - default 10', result => ['session alias or id', 'eventname'], debug => ['session alias or id', 'eventname'], todo => ['list of hashes of jobs todo']
Returns the session id
$kernel->post('session', 'add_tasks', \@tasks); Takes an arraref of hashes, each hash represents one MogileFS::Client method to call and should have the following keys: {method => 'MogileFS::Client method name', domain => 'domain to use', trackers => [array ref of trackers], args => [arrayref, of, args, for, method], taskname => 'name of this task', }
$kernel->post('session', 'shutdown'); Kills the MogileFS session and cleans up.
If result is set in spawn, then your event will get a hashref in ARG0. $_[ARG0]->{status} is whatever is returned from the MogileFS method you called, typically undef means it failed. $_[ARG0]->{task} is the task you originally gave add_task. This should allow you to retry if something fails and it's appropriate to do so (the synopsis contains an example of doing so, using store_content).
Returns back all the warnings and errors MogileFS::Client will spew in ARG0. Mostly just useful for debugging.
This module is kinda simplistic in what MogileFS::Client methods you can use. Essentially all it does is create a new MogileFS::Client object in each child, call the method you provide on that object with the arguments you specified and return the result. Obviously this won't work for any methods that want more than one operation on the same object. For example my $fh = $mogc->newfile( ... ); print $fh 'foobar'; close $fh; isn't going to work. Instead use store_content; The session won't go away until you shutdown. This is to allow you to do things like this: $kernel->post('session', 'add_tasks', \@tasks); $kernel->post('session', 'add_tasks', \@moretasks); and to re-add your tasks from result if they failed.
mock, <mock at obscurity.org>
<mock at obscurity.org>
Please report any bugs or feature requests to bug-poe-component-client-mogilefs at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=POE-Component-Client-MogileFS. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
bug-poe-component-client-mogilefs at rt.cpan.org
You can find documentation for this module with the perldoc command.
perldoc POE::Component::Client::MogileFS
You can also look for information at:
AnnoCPAN: Annotated CPAN documentation
http://annocpan.org/dist/POE-Component-Client-MogileFS
CPAN Ratings
http://cpanratings.perl.org/d/POE-Component-Client-MogileFS
RT: CPAN's request tracker
http://rt.cpan.org/NoAuth/Bugs.html?Dist=POE-Component-Client-MogileFS
Search CPAN
http://search.cpan.org/dist/POE-Component-Client-MogileFS
Copyright 2007 RJ Research, Inc, all rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install POE::Component::Client::MogileFS, copy and paste the appropriate command in to your terminal.
cpanm
cpanm POE::Component::Client::MogileFS
CPAN shell
perl -MCPAN -e shell install POE::Component::Client::MogileFS
For more information on module installation, please visit the detailed CPAN module installation guide.