Programming and Algorithms: Week 21


Web Crawling

What are we doing this week?

This week we are going to look at how to open URLs and join URLs in PYTHON. We are also going to look at the code for a WEB CRAWLER in PYTHON.

Python Python Logo

Powerpoint: Web Crawling

Total running time of videos is 25 minutes.

Python Page Spider Web Crawler Tutorial

Scrape Websites with Python + Beautiful Soup 4 + Requests

Jean Mark Gawron: Introduction to web-crawling in Python

Scrapy - A Fast and Powerful Scraping and Web Crawling Program

HTML Scraping - The Hitchhiker's Guide to Python

Sample Code:
 URL Open * URL Join * Web Crawler

Lab #21
Lab #21 is about adding options to the WEB CRAWLER program.


If you have any suggestions, corrections, or comments, please feel free to e-mail me at: