| Over the course of the past decade, we have all been witnesses to an explosion of information,
in terms of both the amounts of knowledge that exists within the world and the
availability of such information, with the proliferation of the World Wide Web being a
prime example. Although these advancements of knowledge have undoubtedly been
beneficial, they have also created new challenges in information retrieval, in information
processing, and in the extraction of relevant information. This is in part due to a diversity
of file formats as well as the proliferation of loosely structured formats, such as HTML.
The solution to such information retrieval and extraction problems has been to develop
specialized parsers to conduct these tasks. This book will address these tasks, starting
with the most basic principles of data parsing.
The book will begin with an introduction to parsing basics using Perl’s regular expression
engine. Once these regex basic are mastered, the book will introduce the concept of
generative grammars and the Chomsky hierarchy of grammars. Such grammars form the
base set of rules that parsers will use to try to successfully parse content of interest, such as
text or XML files. Once grammars are covered, the book proceeds to explain the two basic
types of parsers—those that use a top-down approach and those that use a bottom-up
approach to parsing. Coverage of these parser types is designed to facilitate the understanding
of more powerful parsing modules such as Yapp (bottom-up) and RecDescent
(top-down).
Once these powerful and flexible generalized parsing modules are covered, the book
begins to delve into more specialized parsing modules such as parsing modules designed
to work with HTML. Within Chapter 6, the book also provides an overview of the LWP modules,
which facilitate access to documents posted on the Web. The parsing examples within
this chapter will use the LWP modules to parse data that is directly accessed from the Web.
Next the book examines the parsing of XML data, which is a markup language that is
increasingly growing in popularity. The XML coverage also discusses SOAP and XML-RPC,
which are two of the most popular methods for accessing remote XML-formatted data. The
book then covers several smaller parsing modules, such as an RSS parser and a date/time
parser, as well as some useful parsing tasks, such as the parsing of configuration files. Lastly,
the book introduces data mining. Data mining provides a means for individuals to work
with extracted data (as well as other types of data) so that the data can be used to learn
more about a given area or to make predictions about future directions that area of interest
may take. This content aims to demonstrate that although parsing is often a critical data
extraction and retrieval task, it may just be a component of a larger data mining system. |
