In the thick of things & my web scraping project from this week
Hi there everyone!
I’m close to being done with my first semester of grad school and what a whirlwhind it’s been! Until August, my programming skills were limited to having completed miniature HTML courses and wrapping ‘Hello, world!’
in the right <tags>. Just four months in, I know: Python, SQL, web scraping with Selenium and BeautifulSoup! It’s been busy.
Just this week, I scraped data from the Texas Department of Licensing and Regulation’s website and from Chicago’s building permit and inspections database. I created a pretend browser window using a webdriver with Selenium and grabbed details for specific cases, put them in a dataframe to analyze them and learned how to scale and automate this whole process! Here’s my code on GitHub if you’re curious 😅
It was fun and frustrating, but the possibilities and the kind of analyses I can do now is very, very encouraging!
Onward!
Here’s some cool data resources and reads I found recently:
Visual Capitalist: The Problem With Our Maps was an interesting read about the falacies in modern maps!
Data Commons: Visualizing the Accumulation of Human-Made Mass on Earth
Old read, but relevent nonetheless: I Cut the 'Big Five' Tech Giants From My Life. It Was Hell
That’s all for now. I’m really excited to share my end-of-semester project in two weeks (that’s keeping me extremely busy right now). Have a cheery weekend! And, I know we’ve all exclaimed at it being December 2021 already, but, I wanted to add to that — take care of yourself. Soak in some winter sun, eat lots of fruits and don’t forget to get some much deserved rest 🫂