阅读BeautifulSoup笔记

来源：互联网发布：怎么查看自己淘宝店铺编辑：程序博客网时间：2024/06/06 09:06

以下代码来自文档，我只是整理一些我当前用到的，用到其他的再添加进来
文档网址：https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/

1、使用BeautifulSoup解析HTML代码,能够得到一个 BeautifulSoup 的对象,并使用方法prettify()能按照标准的缩进格式的结构输出

html_doc = """<html><head><title>The Dormouse's story</title></head><body><p class="title"><b>The Dormouse's story</b></p><p class="story">Once upon a time there were three little sisters; and their names were<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;and they lived at the bottom of a well.</p><p class="story">...</p>"""soup = BeautifulSoup(html_doc)print(soup.prettify())

其中，若只是soup = BeautifulSoup(html_doc)会给warming
改成soup = BeautifulSoup(html_doc，’html.parser’) warming消失
这是标准库里的一个解析器，详情参考文档

2、几个简单的浏览结构化数据的方法

soup.title# <title>The Dormouse's story</title>soup.title.name# u'title'soup.title.string# u'The Dormouse's story'soup.title.parent.name# u'head'soup.p# <p class="title"><b>The Dormouse's story</b></p>soup.p['class']# u'title'soup.a# <a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>soup.find_all('a')#返回一个list# [<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>,#  <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>,#  <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>]soup.find(id="link3")# <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

3、从文档中找到所有标签的链接:

for link in soup.find_all('a'):    print(link.get('href'))    # http://example.com/elsie    # http://example.com/lacie    # http://example.com/tillie

4、从文档中获取所有文字内容:

print(soup.get_text())# The Dormouse's story## The Dormouse's story## Once upon a time there were three little sisters; and their names were# Elsie,# Lacie and# Tillie;# and they lived at the bottom of a well.## ...

0 0