🟡 Advanced Python · Lesson 28

Web Scraping Basics

What are Web Scraping Basics?

Web Scraping Basics means web scraping extracts information from web pages. Always follow website rules and use scraping ethically.

In real programs, this topic helps in reading public web pages responsibly. Learn the idea first, then type the program yourself and compare the output.

💡 At a Glance

Point	Details
Course Area	Advanced Python Professional concepts used to make code reusable, clean and project-ready.
Main Use	reading public web pages responsibly
Example File	`web-scraping-basics.py`
Practice Focus	Run, change values, and explain the output line by line.

Why should you learn this?

It is useful for reading public web pages responsibly.
It connects with extracting HTML tables.
It improves your ability to read, write and debug Python programs.

Important Terms

These terms are used directly in this lesson. Understand them before memorising the code.

Term	Meaning
HTML	Markup language used to create web pages.
BeautifulSoup	Python library commonly used to parse HTML.
select	select is an important term in this topic.
ethics	ethics is an important term in this topic.
robots.txt	File that tells crawlers which pages may be accessed.

Syntax / Basic Pattern

The simple pattern is: prepare data, apply the concept, then show the result.

Basic Pattern

from bs4 import BeautifulSoup
html_text = """
<html><body>
<h1>Python Course</h1>
<p class="price">Free</p>
</body></html>
"""
soup = BeautifulSoup(html_text, "html.parser")

Complete Example Program

Python – web-scraping-basics.py

from bs4 import BeautifulSoup

html_text = """
<html><body>
<h1>Python Course</h1>
<p class="price">Free</p>
</body></html>
"""

soup = BeautifulSoup(html_text, "html.parser")
print(soup.find("h1").text)
print(soup.select_one(".price").text)

Expected Output

Python Course Free

Program Explanation

from bs4 import BeautifulSoup imports ready-made features from a module/library.
html_text = """ stores a value in html_text.
<html><body> performs the next step of the program logic.
<h1>Python Course</h1> performs the next step of the program logic.
<p class="price">Free</p> stores a value in <p class.
</body></html> performs the next step of the program logic.
""" performs the next step of the program logic.

Where will you use it?

Reading public web pages responsibly.
Extracting html tables.
Automating repetitive browsing.

Common Mistakes

Making code complex when a simple function or class is enough.
Not handling possible errors or edge cases.
Mixing project dependencies instead of using a virtual environment.

Practice Tasks

Type the program in web-scraping-basics.py and run it.
Change input values or sample data and observe the new output.
Create one example related to reading public web pages responsibly.
Write 5 lines explaining the logic in your own words.

Summary

Web Scraping Basics is not a theory-only topic. You should be able to explain the meaning, write the example, run it successfully, and use it in a small practical program.