elasticsearch 去重
来源:互联网 发布:怎么永久关闭淘宝店铺 编辑:程序博客网 时间:2024/06/03 08:37
ES字段折叠(collapse)可以去重,但是不hits里面的总数是不去重的,而聚合的cardinality方法去重,会有误差,默认30000之内的数据没有误差,参数precisionThresholdecisionThreshold最大可以调到40000.
DSL语句
{
"query": {
"bool": {
"must": [
{
"term" : {
"invaliddata.keyword" : {
"value" : "0",
"boost" : 1.0
}
}
},
{
"wildcard": {
"customername.keyword": "*周秀芳*"
}
}
]
}
},
"aggs": {
"customerid": {
"cardinality": {
"field": "customerid.keyword"
}
}
},
"collapse":{
"field":"customerid.keyword",
"inner_hits": {
"name": "top_rated",
"size": 1
}
},
"size": 50
}
结果
{"took": 772,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 22,
"max_score": null,
"hits": [
{
"_index": "mtd_customers",
"_type": "mtd_customers_type",
"_id": "4B418955D0766B53E0530B0A420AC4DC_TACT",
"_score": 1.0744119,
"_source": {
"firstname": null,
"birthdate": "1980-10-15T16:00:00.000Z",
"occupation": null,
"identitynumber": "33022419801016",
"customercondition": "0",
"source": "TACT",
"credate": null,
"icropid": null,
"isttime": "2017-11-30T11:53:41.000Z",
"dealerspaceid": null,
"invaliddata": "0",
"customercagegory": null,
"nickname": null,
"respectcode": null,
"id": "4651281237",
"customername": "周秀芳",
"customertype": "1",
"buscleanflag": 1,
"incomamount": null,
"userfullname": null,
"identitytype": "1",
"sex": "2",
"deltime": null,
"middlename": null,
"createuser": null,
"lastname": null,
"ity": null,
"buytel": "135866",
"maritalstatus": "4",
"upttime": "2017-11-30T12:12:40.000Z",
"identitytext": null,
"customerid": "4B418955D0766B53E0530B0A420AC4DC",
"customerflag": 2,
"updatetime": "2017-06-22T14:13:41.000Z",
"corporatename": null,
"respectname": null,
"updateuser": null
},
"fields": {
"customerid.keyword": [
"4B418955D0766B53E0530B0A420AC4DC"
]
},
"inner_hits": {
"top_rated": {
"hits": {
"total": 2,
"max_score": 1.0744119,
"hits": [
{
"_index": "mtd_customers",
"_type": "mtd_customers_type",
"_id": "4B418955D0766B53E0530B0A420AC4DC_TACT",
"_score": 1.0744119,
"_source": {
"firstname": null,
"birthdate": "1980-10-15T16:00:00.000Z",
"occupation": null,
"identitynumber": "33022419801016",
"customercondition": "0",
"source": "TACT",
"credate": null,
"icropid": null,
"isttime": "2017-11-30T11:53:41.000Z",
"dealerspaceid": null,
"invaliddata": "0",
"customercagegory": null,
"nickname": null,
"respectcode": null,
"id": "4651281237",
"customername": "周",
"customertype": "1",
"buscleanflag": 1,
"incomamount": null,
"userfullname": null,
"identitytype": "1",
"sex": "2",
"deltime": null,
"middlename": null,
"createuser": null,
"lastname": null,
"ity": null,
"buytel": "13586653875",
"maritalstatus": "4",
"upttime": "2017-11-30T12:12:40.000Z",
"identitytext": null,
"customerid": "4B418955D0766B53E0530B0A420AC4DC",
"customerflag": 2,
"updatetime": "2017-06-22T14:13:41.000Z",
"corporatename": null,
"respectname": null,
"updateuser": null
}
}
]
}
}
}
}
,
"aggregations": {
"customerid": {
"value": 20
}
}
}
总数用aggregations中的value,结果列表用hits
阅读全文
0 0
- elasticsearch查询去重
- elasticsearch查询去重
- elasticsearch 去重
- elasticsearch去重计数(distinct)
- elasticsearch去重计数(distinct)
- elasticsearch去重计数(distinct、cardinality)
- Elasticsearch的distinct项查询(去重查询)
- 去重
- 去重
- 去重
- 去重
- 去重
- elasticsearch 重启节点
- 数组去重,json数组去重
- 数组去重,list去重
- sql去重
- 词典去重数据
- SQL去重
- vue-awesome-swiper 自动轮播问题
- JAXB その4
- django 原有表格添加或删除 字段
- Mysql设置唯一值
- jvisualvm安装visualgc插件(java.net网站已关闭)
- elasticsearch 去重
- 20. Valid Parentheses
- MFC 通讯录程序思路整理(完成)
- [费用流] ACM-ICPC Asia China-Final Contest J. Mr.Panda and TubeMaster
- java io 概述
- CSV
- 新手入门级的Activity总结(一)
- 网络IPC非阻塞和异步I/O
- ABP 常见错误