Web scraping using Python: wikipedia (API & web crawler bot)

Web scraping using Python and 2 different approaches: custom crawler and using Wikipedia API.

Our first approach is to get information from a website using the Requests and BeautifulSoup4 Python packages. Next to that we will use Wikipedia’s API (MediaWiki) and Wikipedia-API Python library (which is basically wrapper around Wikipedia’s API) to gather data from the Wikipedia website. Also I give additional information about advantages and disadvantages of each approach: web crawler vs API.

Source code: http://bit.ly/2s4bHdB
Requests: http://bit.ly/2s2YaCY
BeautifulSoup4: http://bit.ly/2VjFbBs
Wikipedia API: http://bit.ly/2QkJKrt

Leave a Reply