HTML::Gumbo - HTML5 parser based on gumbo C library
Gumbo is an implementation of the HTML5 parsing algorithm implemented as a pure C99 library with no outside dependencies.
Goals and features of the C library:
Fully conformant with the HTML5 spec.
Robust and resilient to bad input.
Simple API that can be easily wrapped by other languages. (This is one of such wrappers.)
Support for source locations and pointers back to the original text. (Not exposed by this implementation at the moment.)
Relatively lightweight, with no outside dependencies.
Passes all html5lib-0.95 tests.
Tested on over 2.5 billion pages from Google's index.
Beta readiness.
HTML is parsed and re-built from the tree, so tags are balanced (except void elements). Since fragments parsing is not supported at the moment the result always gets html, head and body elements.
No additional arguments for this format.
$html = HTML::Gumbo->new->parse( $html );
HTML::Parser like interface. Pass a sub as callback argument to "parse" method and it will be called for every node in the document:
callback
HTML::Gumbo->new->parse( $html, format => 'callback', callback => sub { my ($event) = shift; if ( $event eq 'document start' ) { my ($doctype) = @_; } elsif ( $event eq 'document end' ) { } elsif ( $event eq 'start' ) { my ($tag, $attrs) = @_; } elsif ( $event eq 'end' ) { my ($tag) = @_; } elsif ( $event eq /^(text|space|cdata|comment)$/ ) { my ($text) = @_; } else { die "Unknown event"; } } );
Note that 'end' events are not generated for void elements, for example hr, br and img.
hr
br
img
No additional arguments except mentioned callback.
Alpha stage.
Produces tree based on HTML::Elements, like HTML::TreeBuilder.
There is major difference from HTML::TreeBuilder, this method produces top level element with tag name 'document' which may have doctype, comments and html tags.
Yes, it's not ready to use as drop in replacement of tree builder. Patches are wellcome. I don't use this formatter at the moment.
The C parser works only with UTF-8, so you have several options to make sure input is UTF-8. First of all define input_is:
input_is
Input is Perl string, for example obtained from "decoded_content" in HTTP::Response. Default value.
Input are octets. Partial implementation of encoding sniffing algorithm is used:
encoding
Use it to hardcode a specific encoding.
UTF-8/UTF-16 BOMs are checked.
encoding_content_type
Encdoning from rransport layer, charset in content-type header.
Not implemented, follow issue 58.
HTML5 defines prescan algorithm that extracts encoding from meta tags in the head.
It would be cool to get it in the C library, but I will accept a patch that impements it in pure perl.
encoding_tentative
The likely encoding for this page, e.g. based on the encoding of the page when it was last visited.
Not implemented. Fragment parsing with or without context is not implemented. Parser also has no origin information, so it wouldn't be implemented.
Not implemented.
Can be implemented using Encode::Detect::Detector. Patches are welcome.
It dies.
utf8
Use utf8 as input_is when you're sure input is UTF-8, but octets. No pre-processing at all. Should only be used on trusted input or when it's preprocessed already.
my $parser = HTML::Gumbo->new;
No options at the moment.
my $res = $parser->parse( "<h1>hello world!</h1>", format => 'tree', input_is => 'string', );
Takes html string and pairs of named arguments:
Output format, default is string. See "SUPPORTED OUTPUT FORMATS".
Whether html is perl 'string', 'octets' or 'utf8' (octets known to be utf8). See "CHARACTER ENCODING OF THE INPUT".
See "CHARACTER ENCODING OF THE INPUT".
Some formatters may have additional arguments.
Return value depends on the picked format.
Ruslan Zakirov <ruz@bestpractical.com>
Under the same terms as perl itself.
To install HTML::Gumbo, copy and paste the appropriate command in to your terminal.
cpanm
cpanm HTML::Gumbo
CPAN shell
perl -MCPAN -e shell install HTML::Gumbo
For more information on module installation, please visit the detailed CPAN module installation guide.