处理python爬虫的timeout报错

来源：互联网发布：kettle java脚本编辑：程序博客网时间：2024/05/18 15:55

尽管添加了

import socketsocket.setdefaulttimeout(timeout)

但是在爬虫过程中依然会出现

Traceback (most recent call last):
File "C:\Users\wenxue5\Documents\Tencent Files\910872628\FileRecv\zongheng.py", line 93, in <module>
search(line.strip())
File "C:\Users\wenxue5\Documents\Tencent Files\910872628\FileRecv\zongheng.py", line 60, in search
the_page = response.read() #读取反馈的内容
File "C:\Python27\lib\socket.py", line 349, in read
data = self._sock.recv(rbufsize)
File "C:\Python27\lib\httplib.py", line 532, in read
return self._read_chunked(amt)
File "C:\Python27\lib\httplib.py", line 590, in _read_chunked
value.append(self._safe_read(chunk_left))
File "C:\Python27\lib\httplib.py", line 634, in _safe_read
chunk = self.fp.read(min(amt, MAXAMOUNT))
File "C:\Python27\lib\socket.py", line 378, in read
data = self._sock.recv(left)
timeout: timed out

这是因为没有对timeout进行处理造成的

import urllib2import socket#In Python 2.7.3try:    urllib2.urlopen("www.baidu.com", timeout = 10)except urllib2.URLError as e:    print type(e)    #not catchexcept socket.timeout as e:    print type(e)    #catched

0 0