Python MongoDB 插入数据,已存在则不执行,不存在则插入

来源:互联网 发布:淘宝代办居住证可靠吗 编辑:程序博客网 时间:2024/05/16 08:23

想把QQ日志爬虫(Python)爬下来的日志保存到 MongoDB 里面。 
但 insert 的时候报错:

<code class="hljs mel has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">E11000 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">duplicate</span> key <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">error</span> collection: QQ.Blog index: _id_ dup key: { : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"965464518_1301232446"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

后来知道错误的原因是:插入的数据和已有数据的 ID 重复了。

我想要的是:插入一篇日志,如果该日志(ID)已存在,则不执行(也不更新);如果不存在则插入。 
百度中大部分的答案都是用更新,但如果 ID 存在的话我是想直接不执行的,没有更新已有数据的必要。


解决方案:

update里有个参数 ‘$setOnInsert’ 可以实现”存在则不执行”的功能,可见 $setOnInsert 官方文档。

示例:

起始数据:

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.insert</span>({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span>})> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.find</span>(){ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>


加入相同 ID 的日志:

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.update</span>({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>}, {$setOnInsert:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span>}}, {upsert:true})WriteResult({ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nMatched"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nUpserted"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nModified"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> })>> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.find</span>(){ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>


加入不同 ID 的日志:

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.update</span>({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123"</span>}, {$setOnInsert:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span>}}, {upsert:true})WriteResult({ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nMatched"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nUpserted"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nModified"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123"</span> })>> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.find</span>(){ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span>{ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>


如果某些内容想更新的话(例如更新 title )可以用 ‘$set’:

<code class="hljs d has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db.Blog.update({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>}, {$setOnInsert:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span>}, $set:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《New Title》"</span>}}, {upsert:<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>})WriteResult({ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nMatched"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nUpserted"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nModified"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> })>> db.Blog.find(){ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《New Title》{ "</span>_id<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" : "</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">", "</span>blog_cont<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" : "</span>abc123<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">", "</span>othe<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">r" : "</span>hello world!<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" }</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>


希望对你有所帮助!



转载请注明出处,谢谢!(原文链接:http://blog.csdn.net/Bone_ACE/article/details/50696477)


0 0
原创粉丝点击