Home | Amazing | Today | Tags | Publishers | Years | Account | Search 
Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT


Kickstart your NLP journey by exploring BERT and its variants such as ALBERT, RoBERTa, DistilBERT, VideoBERT, and more with Hugging Face's transformers library

Key Features

  • Explore the encoder and decoder of the transformer model
  • Become well-versed with BERT along with ALBERT, RoBERTa, and DistilBERT
  • Discover how to pre-train and fine-tune BERT models for several NLP tasks

Book Description

BERT (bidirectional encoder representations from transformer) has revolutionized the world of natural language processing (NLP) with promising results. This book is an introductory guide that will help you get to grips with Google's BERT architecture. With a detailed explanation of the transformer architecture, this book will help you understand how the transformer's encoder and decoder work.

You'll explore the BERT architecture by learning how the BERT model is pre-trained and how to use pre-trained BERT for downstream tasks by fine-tuning it for NLP tasks such as sentiment analysis and text summarization with the Hugging Face transformers library. As you advance, you'll learn about different variants of BERT such as ALBERT, RoBERTa, and ELECTRA, and look at SpanBERT, which is used for NLP tasks like question answering. You'll also cover simpler and faster BERT variants based on knowledge distillation such as DistilBERT and TinyBERT. The book takes you through MBERT, XLM, and XLM-R in detail and then introduces you to sentence-BERT, which is used for obtaining sentence representation. Finally, you'll discover domain-specific BERT models such as BioBERT and ClinicalBERT, and discover an interesting variant called VideoBERT.

By the end of this BERT book, you'll be well-versed with using BERT and its variants for performing practical NLP tasks.

What you will learn

  • Understand the transformer model from the ground up
  • Find out how BERT works and pre-train it using masked language model (MLM) and next sentence prediction (NSP) tasks
  • Get hands-on with BERT by learning to generate contextual word and sentence embeddings
  • Fine-tune BERT for downstream tasks
  • Get to grips with ALBERT, RoBERTa, ELECTRA, and SpanBERT models
  • Get the hang of the BERT models based on knowledge distillation
  • Understand cross-lingual models such as XLM and XLM-R
  • Explore Sentence-BERT, VideoBERT, and BART

Who this book is for

This book is for NLP professionals and data scientists looking to simplify NLP tasks to enable efficient language understanding using BERT. A basic understanding of NLP concepts and deep learning is required to get the best out of this book.

Table of Contents

  1. A Primer on Transformer Model
  2. Understanding the BERT Model
  3. Getting Hands-On with BERT
  4. BERT variants I - ALBERT, RoBERTa, ELECTRA, and SpanBERT
  5. BERT variants II - Based on knowledge distillation
  6. Exploring BERTSUM for Text Summarization
  7. Applying BERT for Other Languages
  8. Exploring Sentence and Domain Specific BERT
  9. Working with VideoBERT, BART, and more
(HTML tags aren't allowed.)

Twitter Tips, Tricks, and Tweets
Twitter Tips, Tricks, and Tweets
When I wrote the first edition of this book, Twitter was just starting to hit the big time: bemused front page articles in just about every magazine and newspaper in the land wondered just what this Twitter thing was all about; the actor Ashton Kutcher (@aplusk) had just cracked the one million follower mark; and Oprah herself had...
Value Solutions in Cooperative Games
Value Solutions in Cooperative Games

This book introduces new concepts for cooperative game theory, and particularly solutions that determine the distribution of a coalitional surplus among the members of the coalition. It also addresses several generalizations of cooperative game theory. Drawing on methods of welfare economics, new value solutions are derived for...

Fundamentals of Wireless Communication
Fundamentals of Wireless Communication
Wireless communication is one of the most vibrant areas in the communication
field today. While it has been a topic of study since the 1960s,
the past decade has seen a surge of research activities in the area. This is
due to a confluence of several factors. First, there has been an explosive
increase in demand for tetherless

Computational Models of Complex Systems (Intelligent Systems Reference Library)
Computational Models of Complex Systems (Intelligent Systems Reference Library)

Computational and mathematical models provide us with the opportunities to investigate the complexities of real world problems. They allow us to apply our best analytical methods to define problems in a clearly mathematical manner and exhaustively test our solutions before committing expensive resources. This is made possible by assuming...

Cabling: The Complete Guide to Network Wiring, 3rd Edition
Cabling: The Complete Guide to Network Wiring, 3rd Edition

The physical linkages responsible for carrying a company's data continue to be the most neglected components of the typical network—to the extent that nearly 700f all network-related problems result from poor cabling.

In this third edition of a widely acclaimed resource, three networking experts

Digital Photography Hacks : 100 Industrial-Strength Tips & Tools
Digital Photography Hacks : 100 Industrial-Strength Tips & Tools
Going beyond the standard fare of most digital photography books, Digital Photography Hacks shares the knowledge that professional photographers have learned through thousands of shots' worth of experience and years of experimentation. With exquisite, full-color photos throughout, the book presents 100 proven...
©2021 LearnIT (support@pdfchm.net) - Privacy Policy