City College, Fall 2019
Intro to Data Science
Week 11: NLP, Text as Data, and Bayes Rule
November 18, 2019Today's Agenda
- Project Notes
- What is NLP?
- Bag of Words Strategies
- Bayesian Analysis
- Tools and Topics in NLP
- Exam Results
Reminder: First Project Submission Due November 23 by 11:59pm
Due this Saturday:
- An email to the professor containing:
- A creative team name, the names of all members of the team, and a link to the team’s repository on GitHub. The repository need not be public, but access must be granted to the professor (grantmlong).
- An attached csv with predictions against test2.csv. The csv must consist exclusively of 2 columns, header rows with the titles rental_id and predictions, and the 2,000 rental ids and corresponding rent predictions. A example of the required formatting can be found here, with suggested methodology for creating the submission file here.
- A markdown file posted in the project Github repo entitled initial_findings.md containing:
- A 200-300 word explanation of the expected performance of the model in terms of mean squared error and the key features driving the team’s modeling performance.
- A 200-300 word summary outlining the team’s intended strategy to improve the predictions for the final round.
Issues with Multicollinearity
Models with multicollinearity can still be useful.
What is NLP?
Natural language processing (NLP) is a field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human (natural) languages.
- Identify the structure and meaning of words, sentences, texts and conversations.
- Deep understanding of broad language.
Source.
NLP Applications: Quora Questions
NLP Applications: Uber One Click Chat
NLP Applications: Facebook Translation
Was this is a prequel worth making?
The labored second chapter in J.K. Rowling's Harry Potter spinoff series is as cumbersome as its title. "Fantastic Beasts: The Crimes of Grindelwald" is a gangly, overly complicated snooze, a rudderless, magic-free visit into Rowling's world of wizards and wizarding. Even the beasts aren't all that fantastic, and the visual effects aren't either.
The labored second chapter in J.K. Rowling's Harry Potter spinoff series is as cumbersome as its title. "Fantastic Beasts: The Crimes of Grindelwald" is a gangly, overly complicated snooze, a rudderless, magic-free visit into Rowling's world of wizards and wizarding. Even the beasts aren't all that fantastic, and the visual effects aren't either.
How do we turn this into a model?
Bag of Words Strategies
Map words
to feature vectors.
Bag of Words Strategies
Advantages:
- Easy to calculate
- Vectors are interpretable
Disadvantages:
- Word order often contains important context!
- Related: Interactions are hard (but not impossible) to model
Data Science Models
Bey's Bayesian Analysis
Bayes's Rules
Know this and how to apply it before any interview. It can get tricky.
Bayes's Rule and Text
From Bayes's Rule to Naive Bayes
How to calculate?
Make some assumptions.
What Makes Naive Bayes Naive?
The conditional independence assumption: For conditional independence to hold true, we're assuming no words are more likely to appear with each other than any others.
Examples: "hot" and "dog", "ball" and "game", harry" and "potter", "computer" and "science"
Tools and Topics in NLP
Other Applications of Text Analysis:
- Part of Speech Tagging
- Named Entity Recognition
- Dependency Parsing
- Topic Modeling
For more, check out the Stanford parser and named entity recognizer and this interactive topic modeling explorer.
Helpful Open Source Tools
- Sci-kit Learn: Has helpful tools for featurization and basic text analysis.
- NLTK: Python package offering a wide array of functionality, including sentiment analysis, part of speech tagging, and more.
- Gensim: Python package offering a specialized tools in topic modeling and a few other domains.
- Stanford Core NLP: Java library offering a wide array of functionality and state of the art performance. (Easy interface with python available through NLTK).
Exam Results
| Multiple Choice | Short Answer | Exam Total | |
|---|---|---|---|
| Points Possible | 70 | 66 | 136 |
| Mean | 56 | 48 | 104 |
| Median | 57 | 49 | 104 |
| Std Dev | 5.4 | 11.9 | 16.0 |
Answer key available on the course page.
City College, Fall 2019
Intro to Data Science
Week 11: NLP, Text as Data, and Bayes Rule
November 18, 2019