Home | Amazing | Today | Tags | Publishers | Years | Account | Search 
Web Scraping with Python (Community Experience Distilled)

Buy

Successfully scrape data from any website with the power of Python

About This Book

  • A hands-on guide to web scraping with real-life problems and solutions
  • Techniques to download and extract data from complex websites
  • Create a number of different web scrapers to extract information

Who This Book Is For

This book is aimed at developers who want to use web scraping for legitimate purposes. Prior programming experience with Python would be useful but not essential. Anyone with general knowledge of programming languages should be able to pick up the book and understand the principals involved.

What You Will Learn

  • Extract data from web pages with simple Python programming
  • Build a threaded crawler to process web pages in parallel
  • Follow links to crawl a website
  • Download cache to reduce bandwidth
  • Use multiple threads and processes to scrape faster
  • Learn how to parse JavaScript-dependent websites
  • Interact with forms and sessions
  • Solve CAPTCHAs on protected web pages
  • Discover how to track the state of a crawl

In Detail

The Internet contains the most useful set of data ever assembled, largely publicly accessible for free. However, this data is not easily reusable. It is embedded within the structure and style of websites and needs to be carefully extracted to be useful. Web scraping is becoming increasingly useful as a means to easily gather and make sense of the plethora of information available online. Using a simple language like Python, you can crawl the information out of complex websites using simple programming.

This book is the ultimate guide to using Python to scrape data from websites. In the early chapters it covers how to extract data from static web pages and how to use caching to manage the load on servers. After the basics we'll get our hands dirty with building a more sophisticated crawler with threads and more advanced topics. Learn step-by-step how to use Ajax URLs, employ the Firebug extension for monitoring, and indirectly scrape data. Discover more scraping nitty-gritties such as using the browser renderer, managing cookies, how to submit forms to extract data from complex websites protected by CAPTCHA, and so on. The book wraps up with how to create high-level scrapers with Scrapy libraries and implement what has been learned to real websites.

Style and approach

This book is a hands-on guide with real-life examples and solutions starting simple and then progressively becoming more complex. Each chapter in this book introduces a problem and then provides one or more possible solutions.

(HTML tags aren't allowed.)

Access Data Analysis Cookbook
Access Data Analysis Cookbook
If you have large quantities of data in a Microsoft Access database, and need to study that data in depth, this book is a data cruncher's dream. Access Data Analysis Cookbook offers practical recipes to solve a variety of common problems that users have with extracting Access data and performing calculations on it. Each recipe includes a...
Genetic Programming Theory and Practice IV (Genetic and Evolutionary Computation)
Genetic Programming Theory and Practice IV (Genetic and Evolutionary Computation)
Genetic Programming Theory and Practice IV was developed from the fourth workshop at the University of Michigan’s Center for the Study of Complex Systems to facilitate the exchange of ideas and information related to the rapidly advancing field of Genetic Programming (GP). Contributions from the foremost international...
Semantic Web Services: Concepts, Technologies, and Applications
Semantic Web Services: Concepts, Technologies, and Applications
In just a few years, service-oriented architectures (SOA) and Web services not only gained considerable interest in computer science research, they were also taken up with unanimity by all major international players in the IT industry. However, and in spite of all existing standards, in most SOA applications much human intervention is still...

Wireless Mesh Networks: Architectures and Protocols
Wireless Mesh Networks: Architectures and Protocols
The book includes a set of research and survey articles featuring the recent advances in theory and applications of wireless mesh networking technology which will be a significant component in the next generation (e.g., fourth generation) mobile communication networks. Wireless mesh networks consist of mesh clients and mesh routers, where the mesh...
Reasoning About Program Transformations
Reasoning About Program Transformations
The motivation of this text lies in what we believe is the inadequacy of current frameworks to reason about the flow of data in imperative programs. This inadequacy clearly shows up when dealing with the individual side effects of loop iterations. Indeed, we face a paradoxical situation where, on the one hand, a typical program spends most of its...
Reverse Engineering: An Industrial Perspective (Springer Series in Advanced Manufacturing)
Reverse Engineering: An Industrial Perspective (Springer Series in Advanced Manufacturing)
Reverse engineering is the process of discovering the technological principles of an object or component through analysis of its structure and function. Such analysis can then be used to redesign the object very quickly using computer-aided design in concert with rapid-manufacturing processes to produce small numbers of components adapted to the...
©2021 LearnIT (support@pdfchm.net) - Privacy Policy