City College, Fall 2019

Intro to Data Science

Week 12: Unsupervised Learning

November 25, 2019

Today's Agenda
  1. Final Projects
  2. Supervised vs. Unsupervised Learning
  3. Clustering
  4. Topic Modeling
  5. Hacking Visualizations in Python
Projects: Results from Submission 1
RankTeam NameMSEPoints
1GIN849,875+6
2EDS879,847+6
3The Data Scientists880,845+4
4RentAdvisor956,380+4
5Science Data1,039,054+2
6100k Offer1,147,571+2
7Datalicious1,158,716+2
Words of Wisdom
  • Simple models can work well: MSE for demo model = 1,666,532
  • Read instructions carefully
  • Check your work
  • Use your time wisely
Supervised vs. Unsupervised Learning

Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs. It infers a function from labeled training data consisting of a set of training examples.

Unsupervised learning is a branch of machine learning that learns from test data that has not been labeled, classified or categorized. Instead of responding to feedback, unsupervised learning identifies commonalities in the data and reacts based on the presence or absence of such commonalities in each new piece of data.

Clustering

Common Clustering Techniques
  • K-means
  • Mean shift
  • Hierarchical
How would you group these points?

Visualization by Andrey A. Shabalin.
K-Means Clustering
  1. Choose k, a number of clusters.
  2. Pick k starting points.
  3. Assign each point to a cluster based on the closest of the k chosen points.
  4. Calculate the points at the center of each cluster.
  5. Assign each point to a cluster based on the closest of the new center points.
  6. Repeat steps 4 & 5 until convergence.

For more, check out these these help notes from CS221 at Stanford.

K-Means: First Try

Visualization by Andrey A. Shabalin.
K-Means: Second Try

Visualization by Andrey A. Shabalin.
K-Means: Third Try

Visualization by Andrey A. Shabalin.
K-Means: Fourth Try

Visualization by Andrey A. Shabalin.
Demo
Mean Shift Clustering
  1. Arrange windows to cover all points.
  2. Compute the number of points in each frame.
  3. Shift the window to the mean.
  4. Repeat until convergence.

These slides adapted from CS109 at Harvard.

Mean Shift Clustering

Visualization by David Sheehan.
Hierarchical Clustering
  1. Each observation starts in its own cluster.
  2. The two closest pairs form a cluster.
  3. Clusters are merged as one moves up the hierarchy.
  4. Repeat until all points belong to the same cluster.

See Wikipedia for more.

Hierarchical Clustering

Visualization by David Sheehan.
Clustering Applications
  • Google Image Search Categories
  • Author Clustering
  • Picking Locations for Hospitals, Police Stations, etc.
  • Outlier Detection

These slides adapted from CS109 at Harvard.
Clustering Application Example

Topic Modeling

Topic modeling provides methods for automatically organizing, understanding, searching, and summarizing large electronic archives.


  • Discover the hidden themes that pervade the collection.
  • Annotate the documents according to those themes.
  • Use annotations to organize, summarize, and search the texts.

This slide adapted from Columbia's David Blei.

Latent Dirichlet Process for Topic Modeling

This slide adapted from Columbia's David Blei.

What topics does this passage cover?

How many genes does an organism need to survive? Last week at the genome meeting here, two genome researchers with radically different approaches presented complementary views of the basic genes needed for life. One research team, using computer analyses to compare known genomes, concluded that today's organisms can be sustained with just 250 genes, and that the earliest life forms required a mere 128 genes. The other researcher mapped genes in a simple parasite and estimated that for this organism, 800 genes are plenty to do the job - but that anything short of 100 wouldn't be enough.


This slide adapted from Columbia's David Blei.

Assigning Topics Via Machine Learning

This slide adapted from Columbia's David Blei.

Hacking Visualizations in Python

Using Altair to Create Interactive d3.js Visualizations
Topic Modeling is Hard!

We'll use Gensim to build our topic model and pyLDAvis to visualize it.

Now for some code.