POE::Component::Amazon::S3 - Work with Amazon S3 using POE
use POE qw(Component::Amazon::S3); POE::Component::Amazon::S3->spawn( alias => 's3', aws_access_key_id => 'your S3 id', aws_secret_access_key => 'your S3 key', ); ### Methods for working with buckets # List buckets, posts back to buckets_done with the result $kernel->post( s3 => 'buckets', 'buckets_done', ); # Add a bucket $kernel->post( s3 => 'add_bucket', 'add_bucket_done', { bucket => 'my-bucket', } ); # Delete a bucket, must be empty of all keys $kernel->post( s3 => 'delete_bucket', 'delete_bucket_done', { bucket => 'my-bucket', } ); # Set access control on a bucket, see below for more info about ACL $kernel->post( s3 => 'set_acl', 'set_acl_done', { bucket => 'my-bucket', acl_short => 'public-read', } ); # Get the access control list for a bucket $kernel->post( s3 => 'get_acl', 'get_acl_done', { bucket => 'my-bucket', } ); ### Methods for working with keys # Add a key with inline data $kernel->post( s3 => 'add_key', 'add_key_done', { bucket => 'my-bucket, key => 'my-inline-key', data => 'testing 123', } ); # Add a key with data from a file $kernel->post( s3 => 'add_key', 'add_key_done', { bucket => 'my-bucket, key => 'my-file-key', file => '/path/to/large_file', } ); # List some keys, used for pagination $kernel->post( s3 => 'list_bucket', 'list_bucket_done', { bucket => 'my-bucket', 'max-keys' => 10, }, ); # List all keys, may make multiple calls internally to list_bucket $kernel->post( s3 => 'list_bucket_all', 'list_bucket_all_done', { bucket => 'my-bucket', }, ); # Get a key, saving the contents in memory $kernel->post( s3 => 'get_key', 'get_key_done', { bucket => 'my-bucket' key => 'my-inline-key', }, ); # Get a key, saving directly to a file $kernel->post( s3 => 'get_key', 'get_key_done', { bucket => 'my-bucket' key => 'my-file-key', file => '/tmp/my-file-key', }, ); # Get only the headers for a key $kernel->post( s3 => 'head_key', 'head_key_done', { bucket => 'my-bucket', key => 'my-inline-key', }, ); # Delete a key $kernel->post( s3 => 'delete_key', 'delete_key_done', { bucket => 'my-bucket', key => 'my-inline-key', }, ); # Set access control on a key, see below for more info about ACL $kernel->post( s3 => 'set_acl', 'set_acl_done', { bucket => 'my-bucket', key => 'my-inline-key', acl_short => 'public-read', } ); # Get the access control list for a key $kernel->post( s3 => 'get_acl', 'get_acl_done', { bucket => 'my-bucket', key => 'my-inline-key', } ); ### Return values # All methods post back to the given state with the same parameters, # return and response. Example: sub add_bucket_done { my ( $kernel, $return, $response ) = @_[ KERNEL, ARG0, ARG1 ]; # $return contains only the results of the call # $response contains the full HTTP::Response object from the call # See individual method documentation below for details on $return }
POE::Component::Amazon::S3 is an asynchronous Amazon S3 client based loosely on Net::Amazon::S3.
Amazon provides an "infinite" Simple Storage Service (S3) where you may store as much data as you like, paying only for the bandwidth and disk space used. An S3 account may contain up to 100 "buckets", each of which may contain any number of keys. Each key can contain any data up to 5GB in size.
To find out more about S3, please visit: http://s3.amazonaws.com/
spawn takes the following named parameters:
spawn
Optional. Sets the alias to which you can post events. This defaults to 's3' if not specified.
Required. Enter your Amazon ID which you receive after signing up for an S3 account.
Required. Enter your Amazon access key.
Optional. If you'd like to communicate with S3 using SSL, set secure to 1. By default all communication is done over HTTP. Enabling this option requires the module POE::Component::SSLify.
secure
Shuts down the component and all subcomponents.
All requests posted to Amazon::S3 take 2 parameters:
The name of an event in the calling session where responses will be sent.
Required by most events, this is a hashref of various options. All events support an optional key pass which takes an arrayref containing anything to be passed-through to the response event.
pass
All responses sent back contain at least 2 parameters:
The return value from the event. This may be a simple boolean value indicating success or failure, a hashref of keys, etc.
The complete HTTP::Response object returned by the request. If the return value returned false, the Amazon S3 error information will be stored in $response->{s3_error}
Anything sent in the pass arrayref will be returned in ARG2, ARG3, etc.
Retrieve a list of all buckets.
$kernel->post( s3 => 'buckets', 'buckets_done', { pass => [ @args ], } );
Returns 0 on failure and a hashref on success:
{ owner_id => $owner_id, owner_displayname => $display_name, buckets => [ { bucket => $bucket_name, creation_date => $date, }, ... ] }
Add a new bucket. Note that there is a limit of 100 buckets per account.
$kernel->post( s3 => 'add_bucket', 'add_bucket_done', # event where response is sent { bucket => $bucket_name, # new bucket to create acl_short => $canned_acl, # optional ACL for bucket, see below pass => [ @args ], # optional items passed through to response event } );
Returns 1 on success and 0 on error.
Delete a bucket. The bucket must not contain any keys or the call will fail.
$kernel->post( s3 => 'delete_bucket', 'delete_bucket_done', { bucket => $bucket_name, # bucket to delete pass => [ @args ], # optional pass-through items } );
Add a key to a bucket. An unlimited number of keys can be added to any one bucket. Each keky may contain any data up to 5GB in size.
$kernel->post( s3 => 'add_key', 'add_key_done', { bucket => $bucket_name, # bucket which will contain the new key key => $key_name, # new key acl_short => $canned_acl, # optional ACL for bucket, see below pass => [ @args ], # optional pass-through items # The key's data can be set from either an in-memory variable or # from a file on disk. Using a disk file is highly recommended for # large items to save on memory usage. data => $inline_data, file => $file_path, } );
Retrieve only the HTTP headers associated with a key.
$kernel->post( s3 => 'head_key', 'head_key_done', { bucket => $bucket_name, key => $key, pass => [ @args ], } );
Retrieve a list of keys in a bucket. This method is best used for paging through many results. If you simply want a list of all keys regardless of how many there are, call list_bucket_all instead.
$kernel->post( s3 => 'list_bucket', 'list_bucket_done', { bucket => $bucket_name, pass => [ @args ], # These optional params are explained below. prefix => $prefix, delimiter => $delimiter, 'max-keys' => $max_keys, marker => $marker, } );
If specified, restricts the response to only contain results that begin with the specified prefix.
If this optional, Unicode string parameter is included with your request, then keys that contain the same string between the prefix and the first occurrence of the delimiter will be rolled up into a single result element and returned in the common_prefixes list. These rolled-up keys are not returned elsewhere in the response. For example, with prefix="USA/" and delimiter="/", the matching keys "USA/Oregon/Salem" and "USA/Oregon/Portland" would be summarized in the response as a single "USA/Oregon" element in the common_prefixes list. If an otherwise matching key does not contain the delimiter after the prefix, it appears in the normal list of keys.
delimiter
common_prefixes
Each element in the common_prefixes list counts as one against the max-keys limit. The rolled-up keys represented by each common_prefixes element do not. If the delimiter parameter is not present in your request, keys in the result set will not be rolled-up and neither the common_prefixes list nor the next_marker element will be present in the response.
max-keys
next_marker
This optional argument limits the number of results returned in response to your query. Amazon S3 will return no more than this number of results, but possibly less. Even if max-keys is not specified, Amazon S3 will limit the number of results in the response (usually this limit is 1000 keys). Check the is_truncated flag to see if your results are incomplete. If so, use the marker parameter to request the next page of results. For the purpose of counting max-keys, a 'result' is either a single key, or a delimited prefix in the common_prefixes list. So for delimiter requests, max-keys limits the total number of list results, not just the number of keys.
is_truncated
marker
This optional parameter enables pagination of large result sets. marker specifies where in the result set to resume listing. It restricts the response to only contain results that occur alphabetically after the value of marker. To retrieve the next page of results, use the last key from the current page of results as the marker in your next request.
See also next_marker, below.
If marker is omitted, the first page of results is returned.
Returns 0 on error and a hashref of results on success:
{ bucket => $bucket_name, prefix => $bucket_prefix, common_prefixes => [ $prefix1, $prefix2, ... ] marker => $bucket_marker, next_marker => $bucket_next_available_marker, max_keys => $bucket_max_keys, is_truncated => $bucket_is_truncated_boolean keys => [ $key1, $key2, ... ] }
If list_bucket was requested with a delimiter, common_prefixes will contain a list of prefixes matching that delimiter. Drill down into these prefixes by making another request with the prefix parameter.
prefix
A convenience element, useful when paginating with delimiters. The value of next_marker, if present, is the largest (alphabetically) of all key names and all common_prefixes in the response. If the is_truncated flag is set, request the next page of results by setting marker to the value of next_marker. This element is only present in the response if the delimiter parameter was sent with the request.
This flag indicates whether or not all results of your query were returned in this response. If your results were truncated, you can make a follow-up paginated request using the marker parameter to retrieve the rest of the results.
Retrieve a list of all keys in a bucket. This may make multiple requests to list_bucket behind the scenes.
$kernel->post( s3 => 'list_bucket_all', 'list_bucket_all_done', { bucket => $bucket_name, pass => [ @args ], } );
Returns 0 on error and a hashref of results on success. This hashref is the same as the one returned by list_bucket.
Retrieve a single key, optionally saving the key's data directly to a file.
$kernel->post( s3 => 'get_key', 'get_key_done', { bucket => $bucket_name, key => $key_name, file => $save_path, # if specified, the key's content is saved # directly to this file. pass => [ @args ], } );
Returns 1 on success and 0 on error. If a file param was not specified, the key's content will be in $response->content().
Delete a single key. WARNING: There is no way to recover a deleted key.
$kernel->post( s3 => 'delete_key', 'delete_key_done', { bucket => $bucket_name, key => $key_name, pass => [ @args ], } );
Every bucket and key in S3 has an access control list. This module provides full support for setting and getting ACLs. For a full explanation of S3's ACLs, please read the technical documentation at http://s3.amazonaws.com/
As mentioned above, the add_bucket and add_key events accept an optional acl_short parameter to set their ACL at the time of creation so set_acl does not need to be called.
add_bucket
add_key
acl_short
set_acl
Set a new ACL on a bucket or key. An ACL may be specified as either one of four standard ACLs, or as a detailed list of users/groups and permissions.
The four canned ACLs you may use with the acl_short param are:
Only the creator of the bucket/key has access.
Anyone may read the bucket/key. If set on a key, it may be downloaded using a standard HTTP GET. This ACL is often used for storing static website content in S3.
Anyone may read and overwrite the bucket/key.
Any other authenticated S3 user may read the bucket/key.
Example using a canned ACL:
$kernel->post( s3 => 'set_acl', 'set_acl_done', { bucket => $bucket_name, key => $key_name, # optional acl_short => 'public-read', pass => [ @args ], } );
ACLs may also be specified as a full list of users and/or groups, and their permissions. You should read the S3 documentation before using this method for setting ACLs.
my $acl = [ # grant WRITE to another S3 user { display_name => $other_name, id => $other_id, permission => 'WRITE', }, # grant READ to all users (same as public-read) { URI => 'http://acs.amazonaws.com/groups/global/AllUsers', permission => 'READ', }, # grant READ to a user with a valid Amazon email account { email => $email_address, permission => 'READ', }, ]; $kernel->post( s3 => 'set_acl', 'set_acl_done', { bucket => $bucket_name, key => $key_name, # optional acl => $acl, pass => [ @args ], } );
Retrieve the full ACL list for a bucket or key.
$kernel->post( s3 => 'get_acl', 'get_acl_done', { bucket => $bucket_name, key => $key_name, # optional pass => [ @args ], } );
Returns an arrayref containing a list of grants on the bucket or key, or 0 on error.
The $return value will be false (0) if an error occurred. If an error occurred, the $response object will contain an additional key, s3_error, which is a hashref of the error response. Example:
s3_error
# $response->{s3_error} contains: { code => 'NoSuchKey', message => 'The resource you requested does not exist', }
For a full list of possible error codes, please see http://docs.amazonwebservices.com/AmazonS3/2006-03-01/ErrorCodeList.html
This module will skip all tests unless a few environment variables are set. Running tests will cost you a very small bit in bandwidth charges. If any tests fail, some buckets and/or keys may not be cleaned up properly, so you should check with a tool like the S3 Firefox Organizer to make sure they are not costing you storage money.
Set to your Amazon S3 ID.
Set to your Amazon S3 Key.
Optional. Set if you want to run the larger stress test that creates 150 keys.
The authors of Net::Amazon::S3, from which much code was borrowed:
Leon Brocard <acme@astray.com>
Brad Fitzpatrick <brad@danga.com>
Andy Grundman <andy@hybridized.org>
Net::Amazon::S3
S3 Firefox Organizer, provides an FTP-like interface - https://addons.mozilla.org/firefox/3247/
This module contains code modified from Amazon that contains the following notice:
This software code is made available "AS IS" without warranties of any kind. You may copy, display, modify and redistribute the software code either by itself or as incorporated into your code; provided that you do not remove any proprietary notices. Your use of this software code is at your own risk and you waive any claim against Amazon Digital Services, Inc. or its affiliates with respect to your use of this software code. (c) 2006 Amazon Digital Services, Inc. or its affiliates.
This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself.
To install POE::Component::Amazon::S3, copy and paste the appropriate command in to your terminal.
cpanm
cpanm POE::Component::Amazon::S3
CPAN shell
perl -MCPAN -e shell install POE::Component::Amazon::S3
For more information on module installation, please visit the detailed CPAN module installation guide.