Skip to content

Implementation of state-of-the-art NLP models using transformers for tasks including machine translation, text-summarization, chatbots, and question answering.

Notifications You must be signed in to change notification settings

Ankit-Kumar-Saini/Coursera_Natural_Language_Processing_Specialization

Repository files navigation

Natural Language Processing Specialization

Table of Contents

  1. Introduction
  2. Instructions to use the repository
  3. My Learnings from the Specialization
  4. Programming Assignments
  5. Results
  6. Disclaimer

Introduction

This Specialization is designed and taught by two experts in NLP, machine learning, and deep learning. Younes Bensouda Mourri is an Instructor of AI at Stanford University who also helped build the Deep Learning Specialization. Łukasz Kaiser is a Staff Research Scientist at Google Brain and the co-author of Tensorflow, the Tensor2Tensor and Trax libraries, and the Transformer paper.

Instructions to use the repository

Using this repository is straight forward. Clone this repository to use. This repository contains all my work for this specialization. All the code base, quiz questions, screenshot, and images, are taken from, unless specified, Natural Language Processing Specialization on Coursera.

My Learnings from the Specialization

In this four course series, I learned about logistic regression and naïve Bayes algorithms and implemented them from scratch in Python, used word vectors to implement sentiment analysis model, complete analogies (King - Man + Woman = Queen), and translate words (English to French), and used locality sensitive hashing for approximate nearest neighbors.

I used dynamic programming, hidden Markov models, and word embeddings to autocorrect misspelled words, autocomplete partial sentences, and identify part-of-speech tags for words.

I learned about dense and recurrent neural networks, LSTMs, GRUs, and Siamese networks in TensorFlow and Trax and used these to perform advanced sentiment analysis, text generation, named entity recognition, and to identify duplicate questions on Quora.

I implemented encoder-decoder (Seq2seq model), causal, and self-attention to perform advanced machine translation of complete sentences, text summarization, question-answering and to build chatbots. State of the art models were taught that includes T5, BERT, transformer, reformer, and more!

Programming Assignments

  1. Course 1: Natural Language Processing with Classification and Vector Spaces

  2. Course 2: Natural Language Processing with Probabilistic Models

    • Week 1: Auto-correct using Minimum Edit Distance

      • Create a simple auto-correct algorithm using minimum edit distance and dynamic programming
    • Week 2: Part-of-Speech (POS) Tagging

      • Learn about Markov chains and Hidden Markov models, then use them to create part-of-speech tags for a Wall Street Journal text corpus. Apply the Viterbi algorithm for POS tagging, which is important for computational linguistics
    • Week 3: N-gram Language Models

      • Learn about how N-gram language models work by calculating sequence probabilities, then build your own autocomplete language model using a text corpus from Twitter (similar models are used for translation, determining the author of a text, and speech recognition)
    • Week 4: Word2Vec and Stochastic Gradient Descent

      • Learn about how word embeddings carry the semantic meaning of words, which makes them much more powerful for NLP tasks, then build your own Continuous bag-of-words model that uses a neural network to compute word embeddings from Shakespeare text.
  3. Course 3: Natural Language Processing with Sequence Models**

  • Week 2: Recurrent Neural Networks for Language Modeling
    • Learn about the limitations of traditional language models and see how RNNs and GRUs use sequential data for text prediction. Build your own next-word generator using a simple RNN on Shakespeare text data.

    • Week 3: LSTMs and Named Entity Recognition

      • Learn about how long short-term memory units (LSTMs) solve the vanishing gradient problem, and how Named Entity Recognition systems quickly extract important information from text. Build your own Named Entity Recognition system using an LSTM and data from Kaggle.
    • Week 4: Siamese Networks

      • Learn about Siamese networks, a special type of neural network made of two identical networks that are eventually merged together. Build your own Siamese network that identifies question duplicates in a dataset from Quora.
  1. Course 4: Natural Language Processing with Attention Models

Results

Some results from the programming assignments of this specialization

  • Neural Machine Translation English to German alt text

  • Word Embeddings projection in 2D space alt text

  • Named Entity Recognition results alt text

  • Auto Suggestion results alt text

  • Duplicate questions identification alt text

  • Text Summarization using Transformers alt text

  • ChatBot results alt text

Disclaimer

The solutions uploaded in this repository are only for reference when you got stuck somewhere. Please don't use these solutions to pass programming assignments.