The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Text-Corpus-VoiceOfAmerica-1.03
River stage zero No dependents
++
/ update_Text_Corpus_VoiceOfAmerica.pl

NAME

update_Text_Corpus_VoiceOfAmerica.pl - Script to update VOA news article corpus.

SYNOPSIS

  update_Text_Corpus_VoiceOfAmerica.pl [-d corpusDirectory -c -t -h]

DESCRIPTION

The script update_Text_Corpus_VoiceOfAmerica.pl may be used to create or update a temporary corpus of Voice of America news articles for personal research and testing of information processing techniques. Read the Voice of America's Terms of Use statement to ensure you abide by it when using this script.

All errors and warnings are logged using Log::Log4perl to the file corpusDirectory/log.txt.

OPTIONS

-d corpusDirectory

The option -d sets the cache directory for the corpus of documents. If the directory does not exist, it will be created. The default is a directory named 'corpus_voa' in the current working directory.

-t

If the option -t is present, parsing tests will be performed on all the documents in the cache.

-v

If the option -v is present, then after each new document is fetched a message is logged stating the number of documents remaining to fetch and the approximate time to completion.

-h

Causes documentation to be printed.

BUGS

This script uses xpath expressions to extract links and text which may become invalid as the format of various pages change, causing a lot of bugs.

Please email bugs reports or feature requests to text-corpus-voiceofamerica@rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Text-Corpus-VoiceOfAmerica. The author will be notified and you can be automatically notified of progress on the bug fix or feature request.

AUTHOR

 Jeff Kubina<jeff.kubina@gmail.com>

COPYRIGHT

Copyright (c) 2009 Jeff Kubina. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

The full text of the license can be found in the LICENSE file included with this module.

KEYWORDS

information processing, english corpus, voa, voice of america

SEE ALSO

Read the Voice of America's Terms of Use statement to ensure you abide by it when using this module.

CHI, Log::Log4perl, Text::Corpus::VoiceOfAmerica, Text::Corpus::VoiceOfAmerica::Document