Post tagged: scraping

mastering BeautifulSoup

tl;dr: BeautifulSoup selectors and code snippets

Once you've become familiar with scraping websites with Python, requests, and BeautifulSoup (if not, read this first), you'll want to start creating reusable components to speed build time and improve data reliability.

Below I've included reference snippets for

  • extracting data
  • cleaning data
  • handling …

how to scrape

tl;dr: Using Python requests and BeautifulSoup libraries to scrape data from most data sources

cost: $0

build time: 30 minutes (MVP) / 120 minutes (cleaner v2)

BeautifulSoup allows you to navigate through the raw HTML pulled down from a website (in JS, this is called 'Walking the DOM'). Generally, you …