Throughout my many years in math education as a tutor, TA, and teacher, one thing that stood out to me was that my students consistently had strong feelings about math. I wondered if this phenomenon extended beyond my students. This led me to the questions, how do people talk about math? And, can I use data to explore topic trends? My expectation going into this project was that I was going to see a lot of hate for math. While that was in fact the case, I also uncovered a few topics that I was not expecting.

Data Collection and Preprocessing

I started by scraping 9,000 tweets that had the keyword “math” using twitter API, tweepy. For preprocessing, I used googletrans to translate any non english tweets. Next I removed any non english characters that did not get translated, converted all the characters to lowercase, and dropped duplicates tweets. Finally, I removed numbers, special characters, and emojis from the tweets. …


The second project in the Data Science bootcamp I am in, Metis, was to make a prediction on a topic of our choosing. The guidelines for the project were to scrape data from a website, and use it to make a prediction on a continuous target variable, using linear regression.

Ironically, choosing a topic was one of the most difficult parts for me. After being introduced to the project, I spent far too much time searching through different websites that seemed reasonable to scrape, looking for some interesting target I could predict. By the end of the second day, I felt it was inevitable that I would have to work on a project I had no interest in, just so I could submit something. Feeling defeated, my mind began to wander and I thought about whether or not my favorite TV show would be renewed. Then it hit me, “Instead of waiting for the announcement, why don’t I use data to predict if it will be renewed or not?” However, a show being renewed or cancelled is not a continuous variable. After talking to a few others in my cohort, the idea of predicting the lifespan of a tv show came up, and my project idea had finally been solidified. …

About

Neda Saleem

I am passionate about leveraging data to develop insights on real world phenomena.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store