Python connects to MongDB
来源:互联网 发布:linux压缩和解压命令 编辑:程序博客网 时间:2024/05/01 07:00
Today, I finished the connection work from Python to MongDB. It uses the module pymongo for purpose.
I'll give the codes directly, because it's very cold here, and I gotto go now...
import reimport jsonimport codecsimport urllib.requestfrom bs4 import BeautifulSoupfrom pymongo import MongoClientdef getHtml(url): page = urllib.request.urlopen(url) html = page.read() return htmldef New(link, dep, son, far): global col url = { 'link': link, 'dep': dep, 'son': son, 'far': far } urllist.append(url) """print(json.dumps(url, sort_keys=True, indent=4, separators=(',', ': ')))""" col.insert(url)def PrintSoup(soup): file = codecs.open('soup.txt', 'w+', 'utf-8') file.write(soup.prettify()) file.close()def Develop(point): url = urllist[point] link = url.get('link') html = getHtml(link) soup = BeautifulSoup(html, 'html.parser') for ai in soup.find_all('a'): href = str(ai.get('href')) if re.match(r'^https?://.+$', href): New(href, url.get('dep') + 1, 0, url.get('link')) url['son'] += 1client = MongoClient('localhost', 27017)db = client.Linkscol = db.Linkspoint = 0urllist = []New("https://image.baidu.com", 1, 0, "")while len(urllist) < 3000: if point >= len(urllist): break Develop(point) point = point + 1
阅读全文
0 0
- Python connects to MongDB
- powershell connects to sqlserver
- gcc+nodejs+python+mongdb+django
- secure CRT can't connects to Ubuntu and Ubuntu can't connets to Internet
- mongdb
- mongdb
- MongDB
- Mongdb
- mongdb
- mongdb
- 34.A user, who is authenticated externally, logs in to a remote machine and connects to the database
- java connects weblogic jms
- 使用 python,批量修改 mongdb 的字段类型
- PHP connects Hostinger MySQL database
- mongdb介绍
- MongDB主从
- MongDB学习
- Mongdb介绍
- 列表
- Jelastic云平台集成Eclipse
- Android 常用的adb命令
- Choice学姐买糖果 二
- android自定义键盘 限制车牌号码输入
- Python connects to MongDB
- C++ primer 薄片系列之 STL 容器和迭代器
- ThreadLocal源码分析
- 利用requests和正则表达式re爬取猫眼电影top100,并下载图片
- CleanMyMac提前嗨购双十一
- linux下重启oracle数据库
- Heat高可用
- 绝对深度解析volatile关键字的实现原理(大部分内容整理于其他博客)
- JS数组中的indexOf方法