html::tokeparser(3)

NAME

HTML::TokeParser - Alternative HTML::Parser interface

SYNOPSIS

require HTML::TokeParser;
$p = HTML::TokeParser->new("index.html")  ||  die  "Can't
open: $!";
while (my $token = $p->get_token) {
    #...
}

DESCRIPTION

The "HTML::TokeParser" is an alternative interface to the
"HTML::Parser" class. It is an "HTML::PullParser" sub
class.

The following methods are available:

$p = HTML::TokeParser->new( $file_or_doc );: The object constructor argument is either a file name,
a file handle object, or the complete document to be
parsed.; If the argument is a plain scalar, then it is taken as
the name of a file to be opened and parsed. If the
file can't be opened for reading, then the constructor
will return an undefined value and $! will tell you
why it failed.; If the argument is a reference to a plain scalar, then
this scalar is taken to be the literal document to
parse. The value of this scalar should not be changed
before all tokens have been extracted.; Otherwise the argument is taken to be some object that
the "HTML::TokeParser" can read() from when it needs
more data. Typically it will be a filehandle of some
kind. The stream will be read() until EOF, but not
closed.
$p->get_token: This method will return the next token found in the
HTML document, or "undef" at the end of the document.
The token is returned as an array reference. The
first element of the array will be a (mostly) single
character string denoting the type of this token: "S"
for start tag, "E" for end tag, "T" for text, "C" for
comment, "D" for declaration, and "PI" for process
instructions. The rest of the array is the same as
the arguments passed to the corresponding HTML::Parser
v2 compatible callbacks (see HTML::Parser). In sum
mary, returned tokens look like this:

["S", $tag, $attr, $attrseq, $text]
["E", $tag, $text]
["T", $text, $is_data]
["C", $text]
["D", $text]
["PI", $token0, $text]; where $attr is a hash reference, $attrseq is an array
reference and the rest is plain scalars.
$p->unget_token($token,...): If you find out you have read too many tokens you can
push them back, so that they are returned the next
time $p->get_token is called.
$p->get_tag( [$tag, ...] ): This method returns the next start or end tag (skip
ping any other tokens), or "undef" if there are no
more tags in the document. If one or more arguments
are given, then we skip tokens until one of the speci
fied tag types is found. For example:

$p->get_tag("font", "/font");; will find the next start or end tag for a font-ele
ment.; The tag information is returned as an array reference
in the same form as for $p->get_token above, but the
type code (first element) is missing. A start tag will
be returned like this:

[$tag, $attr, $attrseq, $text]; The tagname of end tags are prefixed with "/", i.e.
end tag is returned like this:

["/$tag", $text]; $p->get_text( [$endtag] )
This method returns all text found at the current
position. It will return a zero length string if the
next token is not text. The optional $endtag argument
specifies that any text occurring before the given tag
is to be returned. Any entities will be converted to
their corresponding character.

The $p->{textify} attribute is a hash that defines how
certain tags can be treated as text. If the name of a
start tag matches a key in this hash then this tag is
converted to text. The hash value is used to specify
which tag attribute to obtain the text from. If this
tag attribute is missing, then the upper case name of
the tag enclosed in brackets is returned, e.g.
"[IMG]". The hash value can also be a subroutine ref
erence. In this case the routine is called with the
start tag token content as its argument and the return
value is treated as the text.

The default $p->{textify} value is:

{img => "alt", applet => "alt"}

This means that <IMG> and <APPLET> tags are treated as
text, and that the text to substitute can be found in
the ALT attribute.; $p->get_trimmed_text( [$endtag] )
Same as $p->get_text above, but will collapse any
sequences of white space to a single space character.
Leading and trailing white space is removed.

EXAMPLES

This example extracts all links from a document. It will print one line for each link, containing the URL and the textual description between the <A>...</A> tags:: use HTML::TokeParser;
$p = HTML::TokeParser->new(shift||"index.html");; while (my $token = $p->get_tag("a")) {
my $url = $token->[1]{href} || "-";
my $text = $p->get_trimmed_text("/a");
print "$url$text0;; }
This example extract the <TITLE> from the document:: use HTML::TokeParser;
$p = HTML::TokeParser->new(shift||"index.html");
if ($p->get_tag("title")) {
my $title = $p->get_trimmed_text;
print "Title: $title0;; }

COPYRIGHT

This library is free software; you can redistribute it
and/or modify it under the same terms as Perl itself.

docs.sk

comprehensive documentation repository