Detecting Fake News with CNNs & RNNs

As the final project of Brown University’s Deep Learning course, I teamed up with 3 classmates to reimplement the models and reproduce the results from the paper Fake News Identification on Twitter with Hybrid CNN and RNN Models (2018).

We built three models using TensorFlow 2 that classified social media posts (in particular, Tweets), on 5 levels of truthfulness (levels defined by the Liar Dataset):

• an LSTM RNN
• an LSTM with dropout regularization
• an LSTM with a 1D CNN

My role in the project was pair programming with Daniel Kostovetsky to put together the three TensorFlow models. I also added on a live demo component for our project.

To see our final report containing our results, you can download it here. Below is an excerpt from our conclusion.

Determining the truthfulness of a statement without context is a difficult problem. We were only able to achieve a 25.78% accuracy on our dataset, which may seem low, but is close to the paper’s 27% accuracy. For reference, we attempted to classify training examples by hand and achieved an accuracy that was far less than 25.78%, and in fact, no better than random guessing. Many statements in our dataset, such as “The economy bled $24 billion due to the government shutdown,” are virtually impossible to factually evaluate without an external reference. Others, such as “We have a federal government that thinks they have the authority to regulate our toilet seats,” are subjective, opinionated, or don’t have a well-defined truth value. Some statements did contain certain words that made them identifiable as true, or more frequently, false. For instance, the sentence “In the case of a catastrophic event, the Atlanta-area offices of the Centers for Disease Control and Prevention will self-destruct,” includes the word ‘self-destruct’ that is associated with fictional stories, implying that it is false. Perhaps our model learned to make classifications by looking for such words.

You can find the source code of the project at https://github.com/CaoRuiming/gradient-ascent-project.