Data::FlexSerializer - (De-)serialization from/to (compressed) JSON, Storable or Sereal
This module was originally written to convert away from persistent use of Storable to using JSON at Booking.com. Since then mostly due to various issues with JSON not accurately being able to represent Perl datastructures (e.g. preserve encoding flags) we've started to migrate to Sereal::Encoder instead.
However the API of this module is now slightly awkward because now it needs to deal with the possible detection and emission of these three formats, and it still uses the JSON format by default which is no longer the recommended way to use it.
# For all of the below use Data::FlexSerializer;
# We *only* read/write compressed JSON by default: my $strict_serializer = Data::FlexSerializer->new; my @blobs = $strict_serializer->serialize(@perl_datastructures); my @perl_datastructures = $strict_serializer->deserialize(@blobs);
# We can optionally detect compressed JSON as well, will accept # mixed compressed/uncompressed data. This works for all the input # formats. my $lax_serializer = Data::FlexSerializer->new( detect_compression => 1, );
# If we know that all our data is compressed we can skip the # detection step. This works for all the input formats. my $lax_compress = Data::FlexSerializer->new( assume_compression => 1, compress_output => 1, # This is the default );
my $storable_to_json = Data::FlexSerializer->new( detect_compression => 1, # check whether the input is compressed detect_storable => 1, # accept Storable images as input compress_output => 1, # This is the default );
my $storable_to_sereal = Data::FlexSerializer->new( detect_sereal => 1, output_format => 'sereal', );
my $sereal_backcompat = Data::FlexSerializer->new( detect_sereal => 1, # accept Sereal images as input );
my $flex_to_json = Data::FlexSerializer->new( detect_compression => 1, detect_json => 1, # this is the default detect_sereal => 1, detect_storable => 1, output_format => 'sereal', );
my $flex_to_json = Data::FlexSerializer->new( detect_compression => 1, detect_json => 1, # this is the default detect_sereal => 1, detect_storable => 1, output_format => 'sereal', sereal_decoder => Sereal::Decoder->new(...), sereal_encoder => Sereal::Encoder->new(...), );
This simple OO module implements a serializer/deserializer for the basic builtin Perl data structures (no blessed structures, no filehandles, no regexes, no self-referential structures). It can produce JSON, Storable or Sereal images. The output can be zlib-compressed or uncompressed depending on settings.
The deserialization phase is more powerful: Depending on the serializer settings, it can be strict in only accepting compressed or uncompressed JSON, or it can auto-detect zlib compression. Additionally, since the purpose of this is to allow painless migration away from storing Storable images persistently, the deserialization code will optionally detect that the input data is a (compressed or uncompressed) Storable image and handle it gracefully. This flexibility comes at a price in performance, so in order to keep the impact low, the default options are more restrictive, see below.
add_format class method to add support for custom formats.
add_format
Data::FlexSerializer->add_format( data_dumper => { serialize => sub { shift; goto \&Data::Dumper::Dumper }, deserialize => sub { shift; goto \&eval }, detect => sub { $_[1] =~ /\$[\w]+\s*=/ }, } ); my $flex_to_dd = Data::FlexSerializer->new( detect_data_dumper => 1, output_format => 'data_dumper', );
Constructor. Takes named arguments.
assume_compression is a flag that makes the deserialization assume that the data will be compressed. It won't have to guess, making the deserialization faster. Defaults to true.
assume_compression
detect_compression is a flag that also affects only the deserialization step. If set, it'll auto-detect whether the input is compressed. Mutually exclusive with assume_compression. If detect_compression is set, but assume_compression is not explicitly specified, assume_compression will be disabled (where it otherwise defaults to true).
detect_compression
compress_output is a flag indicating whether compressed or uncompressed dumps are to be generated during the serialization. Defaults to true.
compress_output
compression_level is an integer indicating the compression level (0-9).
compression_level
output_format can be either set to the string json (default), storable or sereal. It has the obvious effect. Its value can be changed at runtime via the accessor to facilitate having certain output formats in experiments.
output_format
json
storable
sereal
Note that if you dynamically change this to sereal at runtime the first call to "serialize" after that will dynamically construct a Sereal::Encoder object, to avoid this supply a custom "sereal_encoder" object when constructing the object, and we won't have to construct it dynamically later.
detect_json, if set, forces Data::FlexSerializer into JSON-compatibility mode. Defaults to on.
detect_json
Data::FlexSerializer
detect_storable, if set, forces Data::FlexSerializer into Storable-compatibility mode. Apart from JSON input, it will also detect whether the provided blob is in valid Storable format. Defaults to off.
detect_storable
detect_sereal, if set, forces Data::FlexSerializer into Sereal-compatibility mode. Apart from JSON input, it will also detect whether the provided blob is in valid Sereal format. Defaults to off.
detect_sereal
You can supply sereal_encoder or sereal_decoder arguments with your own Serial decoder/encoder objects. Handy if you want to pass custom options to the encoder or decoder.
sereal_encoder
sereal_decoder
By default we create objects for you at BUILD time. So you don't need to supply this for optimization purposes either.
Given a list of things to serialize, this does the job on each of them and returns a list of serialized blobs.
In scalar context, this will return a single serialized blob instead of a list. If called in scalar context, but passed a list of things to serialize, this will croak because the call makes no sense.
The opposite of serialize, doh.
serialize
Given a (single!) file name, reads the file contents and deserializes them. Returns the resulting Perl data structure.
Since this works on one file at a time, this doesn't return a list of data structures like deserialize() does.
deserialize()
$serializer->serialize_to_file( $data_structure => '/tmp/foo/bar' );
Given a (single!) Perl data structure, and a (single!) file name, serializes the data structure and writes the result to the given file. Returns true on success, dies on failure.
Steffen Mueller <smueller@cpan.org>
Ævar Arnfjörð Bjarmason <avar@cpan.org>
Burak Gürsoy <burak@cpan.org>
Elizabeth Matthijsen <liz@dijkmat.nl>
Caio Romão Costa Nascimento <cpan@caioromao.com>
Jonas Galhordas Duarte Alves <jgda@cpan.org>
This module was originally developed at and for Booking.com. With approval from Booking.com, this module was generalized and put on CPAN, for which the authors would like to express their gratitude.
(C) 2011, 2012, 2013 Steffen Mueller and others. All rights reserved. This code is available under the same license as Perl version 5.8.1 or higher. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
To install Data::FlexSerializer, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Data::FlexSerializer
CPAN shell
perl -MCPAN -e shell install Data::FlexSerializer
For more information on module installation, please visit the detailed CPAN module installation guide.