第一章 初见网络爬虫

urllib是Python的标准库，包含了从网络请求数据，处理Cookie，甚…..本书中广泛使用urllib，所以建议读读这个库的Python文档
urllib官方文档 廖雪峰教程

from urllib.request import urlopenhtml = urlopen("http://www.pythonscraping.com/pages/page1.html");print(html.read());

BeautifulSoup：它通过定位HTML标签来格式化和组织复杂的网络信息。

安装Python的包管理器pip，然后cmd在命令提示符里运行：

pip install beautifulsoup4

运行BeautifulSoup

from urllib.request import urlopenfrom bs4 import BeautifulSouphtml = urlopen("http://www.pythonscraping.com/pages/page1.html");bs0bj = BeautifulSoup(html.read());print(bs0bj.h1)

可能的异常处理：

from urllib.request import urlopenfrom urllib.error import HTTPError,URLErrorfrom bs4 import BeautifulSoupdef getTitle(url):    try:        html=urlopen(url);    except(HTTPError,URLError) as e:        return None;    try:        bs0bj = BeautifulSoup(html.read())        title = bs0bj.body.h1;    except AttributeError as e:        return None;    return title;title=getTitle("http://www.pythonscraping.com/pages/page1.html");if title == None:    print("Title could not be found");else:    print(title);

写爬虫的时候，思考代码的总体格局，让代码可以捕捉异常又容易阅读。