More than Just a Pretty Face?

Jan. 8, 2018, 10:56 p.m.

The Backstory

I'll admit -- I've never really understood the appeal of the show, The Bachelor. Contrived interactions, fake relationships, all filmed on camera. However, during the last season, I would have the show playing in the background while preparing dinner. After watching all the fanfare and the dramatics, it led me to wonder if we can just skip all of the charades. What if it was just about appearances, and the Bachelor might be appealing to much simpler senses. Perhaps there is a special "look" shared across all Bachelor Winners unwittingly pulling at his heartstrings?

The Questions

  1. Can we predict the winner of The Bachelor solely on facial appearances?
  2. What might the winner of the The Bachelor look like generally speaking?

The Data and Conclusions

Part 1:

Before jumping to any conclusions, I want to acknowledge that this isn't the best designed data science experiment. We need to make some rather bold assumptions that I will discuss in the technical sections below. Generally speaking, the approach was to take a set of face images for previous Bachelor Winners and Non Winners. We would then use a Convolutional Neural Network to extract embeddings for both sets of pictures. And then we train a binary classifier on the embeddings that would then predict who the winner might be. The output of the model states a predicted probability that the image is a "Winner".

And here are the results:

Krystal, 69%
Ashley, 67%
Annaliese, 65%
Chelsea, 61%
Jacqueline, 56%
Lauren J, 43%
Lauren G, 42%
Jessica, 38%
Kendall, 35%
Bekah M, 27%
Amber, 26%
Marikh, 21%
Brittane J, 18%
Becca K, 18%
Lauren B, 16%
Caroline, 16%
Ali, 14%
Lauren S., 12%
Bibiana, 11%
Bri, 10%
Seinne, 9%
Jenna, 9%
Brittany T, 9%
Jenny, 8%
Tia, 8%
Valerie, 7%
Maquel, 7%
Nysha, 7%
Olivia, 4%

Part 2:

The issue with neural networks is that the features are extracted using self-generated embeddings. The network learns what features to look out for by itself, and so we're left wondering the appearance of a "Winning Face". While this part diverges distinctly from the approach in Part 1 of the project, I used a technique that applied face averaging for the Winners and Non-Winners. Face averaging takes sets of images and essentially overlays each image on top of each other while correcting for the positioning of the face.

So what does the "Average Face" for the Winners look like?

And what about the "Average Face" for the Non Winners?

And just because I was curious, what does the average contestant for the past few seasons look like?

Season 17
Season 18
Season 19
Season 20
Season 21
Season 22

Average faces are quite frankly... very average. Between seasons, there are very subtle differences if you look carefully.

The Tech and Sources

This project was primarily my initial foray into using tensorflow, keras, opencv, and dlib. These are all machine learning or image manipulation packages from which we can build an analytical pipeline. Keras is a high-level neural-network API that drastically simplifies deep learning. Also, rather than building my own model for facial recognition that would require servers on servers, I implemented transfer learning from the VGG-Face model. At a high level, VGG-Face was trained to pick out facial features that distinguish people from each other and then assign a "score" to the feature. Then a binary classifier learns what feature scores belong to a "Winner" and what scores belong to a "Non Winner".

Now this all only works under the assumption that there is an underlying characteristic shared in common between the Winners. So if the Bachelor is actually less shallow than I presume, then your guess is as good as mine. Also, we'd probably need more training images, more preprocessing, and standardizing for these models. I created a few scripts to download images from google, find the appropriate face (it's actually hard to find pictures of the contestants without the guy), shift the facial features to the center while keeping eye corners, nose corners, lips, etc. all parallel and proportion to each other.

Given the above restrictions, the model isn't nearly as performant when trying to generalize to new faces. I set the dropout to try to control overfitting, but I wanted to keep it at least balanced on accuracy. The validation accuracy was around 71% (quite honestly, much better than I had hoped). It had sensitivity (true positive rate) of around 86% which I believe to mean that if it deemed you as a Winner, it was fairly correct. Still, I'm almost certain that the model suffers from overfitting on the training examples i.e. it can find Amanda Marsh, Emily Maynard, Vanessa Grimaldi, etc. very well but it has trouble deciding on new people.

The face averaging component uses dlib which is a super neat and useful library, but it's probably been the most difficult thing ever to build, compile, and get working. The problem with face averaging is that face images come in different poses, postures, and gazes. If we naively stack the images on top of each other, someone's eye would be in another person's nose, and in another person's ear. Dlib corrects this by detecting 68 facial landmarks, plots it on a coordinate plan, does a similarity transform on the calculated Delaunay triangles, and warps the triangles into the appropriate place. So eyes between images are aligned on top of each other, but the distance between each person's eye and mouth is preserved.

This project has made me realize that I have a ton more to learn about deep learning and all the tools to simplify the process. I played around with Docker, and it and might be something I spin up next time so I won't have to mess around with 5 different versions of Python.

Return to Homepage