Elastic search 系统学习之四： 文档API

一、Index API

1、插入文档

curl -XPUT 'localhost:9200/twitter/tweet/1?pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'
或

http://localhost:5601/app/kibana#/dev_tools/console?load_from=https://www.elastic.co/guide/en/elasticsearch/reference/current/snippets/docs-index_/1.json

2、自动创建索引

索引操作自动创建索引

put操作可以手动创建新类型

动态创建索引禁止：

  action.auto_create_index: false 或 the cluster update settings API


动态映射类型禁止：

  index.mapper.dynamic： true


动态创建索引模式：

     action.auto_create_index:aaa*,-bbb*,+ccc*,-*

      +表示允许，-表示不允许

3、版本控制

curl -XPUT 'localhost:9200/twitter/tweet/1?version=2&pretty' -H 'Content-Type: application/json' -d'
{
    "message" : "elasticsearch now has versioning support, double cool!"
}
'
或
PUT twitter/tweet/1?version=2
{
    "message" : "elasticsearch now has versioning support, double cool!"
}

4、版本控制类型

internal，external or external_gt, external_gte

5、操作类型

op_type: create   存在就会创建失败

PUT twitter/tweet/1?op_type=create
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"

}

或

curl -XPUT 'localhost:9200/twitter/tweet/1?op_type=create&pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

6、自动产生id

POST twitter/tweet/
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

或

curl -XPOST 'localhost:9200/twitter/tweet/?pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

7、路由

POST /twitter/tweet?routing=kimchy&pretty
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

或

curl -XPOST 'localhost:9200/twitter/tweet?routing=kimchy&pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

文档根据routing值将文档路由到相应的分片

8、分布式

分片有主次之分

9、等待分片激活

index.write.wait_for_active_shards: 2  写操作会等待2个分片备份恢复才会返回或者超时

 及时设置为all,

也不一定能保证一定写成功

判断是否激活是在各个备份写入前判断的

10、刷新

       写入后进行刷新

11、等待更新noop update

12、超时

       PUT twitter/tweet/1?timeout=5m
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

或

curl -XPUT 'localhost:9200/twitter/tweet/1?timeout=5m&pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

二、获取

1、查询文档

GET twitter/tweet/1

或

curl -XGET 'localhost:9200/twitter/tweet/0?pretty'

文档存在查询

HEAD /twitter/tweet/1

或

curl -XHEAD 'localhost:9200/twitter/tweet/0?pretty'

2、实时性

realtimeparameter: false 关闭实时性

默认，都是实时的

文档更新后没有refresh，查询会导致refresh，从而使文档可见

3、源过滤  官方教程有毛病，不配套


关闭源检索？

GET twitter/tweet/0?_source=false

或

curl -XGET 'localhost:9200/twitter/tweet/0?_source=false&pretty'

获取部分源信息

GET twitter/tweet/1?_source=user&_source_exclude=messag*

4、存储域信息

设置counter域为非存储域

PUT test
{
   "mappings": {
      "tweet": {
         "properties": {
            "counter": {
               "type": "integer",
               "store": false
            },
            "tags": {
               "type": "keyword",
               "store": true
            }
         }
      }
   }

}

或

curl -XPUT 'localhost:9200/test?pretty' -H 'Content-Type: application/json' -d'
{
   "mappings": {
      "tweet": {
         "properties": {
            "counter": {
               "type": "integer",
               "store": false
            },
            "tags": {
               "type": "keyword",
               "store": true
            }
         }
      }
   }
}
'

PUT test/tweet/1
{
    "counter" : 1,
    "tags" : ["red"]
}

或

curl -XPUT 'localhost:9200/test/tweet/1?pretty' -H 'Content-Type: application/json' -d'
{
    "counter" : 1,
    "tags" : ["red"]
}
'

GET /test/tweet/1?stored_fields=tags,counter

或

curl -XGET 'localhost:9200/test/tweet/1?stored_fields=tags,counter&pretty'

5、直接获取_source

获取文档source信息

GET twitter/tweet/1/_source

或

curl -XGET 'localhost:9200/twitter/tweet/1/_source?pretty'

过滤source信息

GET /twitter/tweet/1/_source?_source_include=user&_source_exclude=message'
或

curl -XGET 'localhost:9200/twitter/tweet/1/_source?_source_include=*.id&_source_exclude=entities'&pretty'

测试文档是否存在

HEAD twitter/tweet/1/_source

或

curl -XHEAD 'localhost:9200/twitter/tweet/1/_source?pretty'

6、路由

GET twitter/tweet/2?routing=user1

7、GET偏向

默认各个备份随机提供服务

_primary， _local

8、刷新

     refresh:true

9、分布式

10、版本控制

三、删除API

1、操作

      GET twitter/tweet/1?routing=user1

2、版本控制

3、路由

      DELETE /twitter/tweet/1?routing=kimchy

4、自动创建索引

      如果删除的文件的索引不存在，delete操作会自动创建索引

5、等待激活的分片

6、刷新

7、超时

     DELETE /twitter/tweet/1?timeout=5m

     或

     curl -XDELETE 'localhost:9200/twitter/tweet/1?timeout=5m&pretty'

四、通过query删除文档API

1、通过query删除文档

POST twitter/_delete_by_query
{
  "query": {
    "match": {
      "message": "trying"
    }
  }

}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "message": "some message"
    }
  }
}
'

2、过程

     获得索引快照--------->删除匹配文档   两个步骤中间发生了文档更新，删除会有版本冲突

3、删除索引所有文档

POST twitter/tweet/_delete_by_query?conflicts=proceed
{
  "query": {
    "match_all": {}
  }

}

或

curl -XPOST 'localhost:9200/twitter/tweet/_delete_by_query?conflicts=proceed&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}
'

4、多索引清空

POST twitter,blog/tweet,post/_delete_by_query
{
  "query": {
    "match_all": {}
  }

}

或

curl -XPOST 'localhost:9200/twitter,blog/tweet,post/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}
'

5、路由限定处理的分片

POST twitter/_delete_by_query?routing=1
{
  "query": {
    "range" : {
        "age" : {
           "gte" : 10
        }
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?routing=1&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range" : {
        "age" : {
           "gte" : 10
        }
    }
  }
}
'
6、限定滚动数

POST twitter/_delete_by_query?scroll_size=5000
{
  "query": {
    "term": {
      "user": "kimchy"
    }
  }

}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?scroll_size=5000&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}
'

7、URL参数

待补充

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html

8、任务接口

获取delete-by-query任务的状态

GET _tasks?detailed=true&actions=*/delete/byquery
或

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*/delete/byquery&pretty'

根据任务id直接查看任务

GET /_tasks/taskId:1

或

curl -XGET 'localhost:9200/_tasks/taskId:1?pretty'

9、取消delete-by-query任务

POST _tasks/task_id:1/_cancel

或

curl -XPOST 'localhost:9200/_tasks/task_id:1/_cancel?pretty'

10、rethrottling

POST _delete_by_query/task_id:1/_rethrottle?requests_per_second=-1

或

curl -XPOST 'localhost:9200/_delete_by_query/task_id:1/_rethrottle?requests_per_second=-1&pretty'

11、人工分片

POST twitter/_delete_by_query
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'
curl -XPOST 'localhost:9200/twitter/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 1,
    "max": 2
  },
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'

确认工作：

GET _refresh
POST twitter/_search?size=0&filter_path=hits.total
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

或

curl -XGET 'localhost:9200/_refresh?pretty'
curl -XPOST 'localhost:9200/twitter/_search?size=0&filter_path=hits.total&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'

12、自动分片

POST twitter/_delete_by_query?refresh&slices=5
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?refresh&slices=5&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'

确认：

POST twitter/_search?size=0&filter_path=hits.total
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

五、更新API

PUT /test1/type1/1
{
    "counter" : 1,
    "tags" : ["red"]
}

1、脚本更新

POST test1/type1/1/_update
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    }
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    }
}
'

添加一个tag

POST test1/type1/1/_update
{
    "script" : {
        "source": "ctx._source.tags.add(params.tag)",
        "lang": "painless",
        "params" : {
            "tag" : "blue"
        }
    }
}
或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "ctx._source.tags.add(params.tag)",
        "lang": "painless",
        "params" : {
            "tag" : "blue"
        }
    }
}
'

添加一个新域

POST test1/type1/1/_update
{
    "script" : "ctx._source.new_field = 'value_of_new_field'"
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : "ctx._source.new_field = \u0027value_of_new_field\u0027"
}
'

删除一个域

POST test1/type1/1/_update
{
    "script" : "ctx._source.remove('new_field')"
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : "ctx._source.remove(\u0027new_field\u0027)"
}
'

条件删除

POST test1/type1/1/_update
{
    "script" : {
        "source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'none' }",
        "lang": "painless",
        "params" : {
            "tag" : "green"
        }
    }
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = \u0027delete\u0027 } else { ctx.op = \u0027none\u0027 }",
        "lang": "painless",
        "params" : {
            "tag" : "green"
        }
    }
}
'

2、局部文档更新

并入到文档中

POST test1/type1/1/_update
{
    "doc" : {
        "name" : "new_name"
    }

}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    }
}
'

3、删除等待更新

不改变任何内容时候返回： result:noop

POST test1/type1/1/_update
{
    "doc" : {
        "name" : "new_name"
    }
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    }
}
'

使noop行为失效

POST test1/type1/1/_update
{
    "doc" : {
        "name" : "new_name"
    },
    "detect_noop": false
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    },
    "detect_noop": false
}
'

4、Upserts

存在就更新，文档不存在就将upsert中的字段插入：

例如：

POST test1/type1/1/_update
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    },
    "upsert" : {
        "counter" : 1
    }
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    },
    "upsert" : {
        "counter" : 1
    }
}
'

5、通过脚本upsert

POST /sessions/session/dh3sgudg8gsrgl/_update
{
    "scripted_upsert":true,
    "script" : {
        "id": "my_web_session_summariser",
        "params" : {
            "pageViewEvent" : {
                "url":"foo.com/bar",
                "response":404,
                "time":"2014-01-01 12:32"
            }
        }
    },
    "upsert" : {}
}

或

curl -XPOST 'localhost:9200/sessions/session/dh3sgudg8gsrgl/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "scripted_upsert":true,
    "script" : {
        "id": "my_web_session_summariser",
        "params" : {
            "pageViewEvent" : {
                "url":"foo.com/bar",
                "response":404,
                "time":"2014-01-01 12:32"
            }
        }
    },
    "upsert" : {}
}
'

6、doc_as_upsert

POST test1/type1/3/_update
{
    "doc" : {
        "name" : "new_name"
    },
    "doc_as_upsert" : true
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    },
    "doc_as_upsert" : true
}
'

六、通过query API更新

1、最简的query 更新api

POST twitter/_update_by_query?conflicts=proceed

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?conflicts=proceed&pretty'

POST twitter/tweet/_update_by_query?conflicts=proceed

或

curl -XPOST 'localhost:9200/twitter/tweet/_update_by_query?conflicts=proceed&pretty'

2、通过query限制范围

POST twitter/_update_by_query?conflicts=proceed
{
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

3、脚本

POST twitter/_update_by_query
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}
'

更新字段：

POST twitter/_update_by_query
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}
'

4、多索引操作

POST twitter,blog/tweet,post/_update_by_query

或

curl -XPOST 'localhost:9200/twitter,blog/tweet,post/_update_by_query?pretty'

POST twitter/_update_by_query?routing=1
或

curl -XPOST 'localhost:9200/twitter/_update_by_query?routing=1&pretty'

POST twitter/_update_by_query?scroll_size=100
或

curl -XPOST 'localhost:9200/twitter/_update_by_query?scroll_size=100&pretty'

5、接待节点属性

PUT _ingest/pipeline/set-foo
{
  "description" : "sets foo",
  "processors" : [ {
      "set" : {
        "field": "foo",
        "value": "bar"
      }
  } ]
}
POST twitter/_update_by_query?pipeline=set-foo

或

curl -XPUT 'localhost:9200/_ingest/pipeline/set-foo?pretty' -H 'Content-Type: application/json' -d'
{
  "description" : "sets foo",
  "processors" : [ {
      "set" : {
        "field": "foo",
        "value": "bar"
      }
  } ]
}
'
curl -XPOST 'localhost:9200/twitter/_update_by_query?pipeline=set-foo&pretty'

6、URL参数

pretty

refresh

wait_for_completion

wait_for_active_shards

timeout

7、任务API执行任务

获取所有执行update-by-query的请求的状态

GET _tasks?detailed=true&actions=*byquery

或

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*byquery&pretty'

通过task id查询task

GET /_tasks/taskId:1

或

curl -XGET 'localhost:9200/_tasks/taskId:1?pretty'

8、取消任务API

POST _tasks/task_id:1/_cancel

或

curl -XPOST 'localhost:9200/_tasks/task_id:1/_cancel?pretty'

9、Rethrottling

POST _update_by_query/task_id:1/_rethrottle?requests_per_second=-1
‘或’

POST _update_by_query/task_id:1/_rethrottle?requests_per_second=-1

10、manual slicing

POST twitter/_update_by_query
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}
POST twitter/_update_by_query
{
  "slice": {
    "id": 1,
    "max": 2
  },
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "script": {
    "source": "ctx._source[\u0027extra\u0027] = \u0027test\u0027"
  }
}
'
curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 1,
    "max": 2
  },
  "script": {
    "source": "ctx._source[\u0027extra\u0027] = \u0027test\u0027"
  }
}
'

确认：

GET _refresh
POST twitter/_search?size=0&q=extra:test&filter_path=hits.total

或

curl -XGET 'localhost:9200/_refresh?pretty'
curl -XPOST 'localhost:9200/twitter/_search?size=0&q=extra:test&filter_path=hits.total&pretty'

11、自动分片

POST twitter/_update_by_query?refresh&slices=5
{
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?refresh&slices=5&pretty' -H 'Content-Type: application/json' -d'
{
  "script": {
    "source": "ctx._source[\u0027extra\u0027] = \u0027test\u0027"
  }
}
'

确认：

POST twitter/_search?size=0&q=extra:test&filter_path=hits.total
或

curl -XPOST 'localhost:9200/twitter/_search?size=0&q=extra:test&filter_path=hits.total&pretty'

12、pick up新属性

PUT test2
{
  "mappings": {
    "test": {
      "dynamic": false,   //只存储在source中不进行索引
      "properties": {
        "text": {"type": "text"}
      }
    }
  }
}

POST test2/test?refresh
{
  "text": "words words",
  "flag": "bar"
}
POST test2/test?refresh
{
  "text": "words words",
  "flag": "foo"
}
PUT test2/_mapping/test    //添加新的flag域
{
  "properties": {
    "text": {"type": "text"},
    "flag": {"type": "text", "analyzer": "keyword"}
  }
}

或

curl -XPUT 'localhost:9200/test?pretty' -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "test": {
      "dynamic": false,   
      "properties": {
        "text": {"type": "text"}
      }
    }
  }
}
'
curl -XPOST 'localhost:9200/test/test?refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "text": "words words",
  "flag": "bar"
}
'
curl -XPOST 'localhost:9200/test/test?refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "text": "words words",
  "flag": "foo"
}
'
curl -XPUT 'localhost:9200/test/_mapping/test?pretty' -H 'Content-Type: application/json' -d'
{
  "properties": {
    "text": {"type": "text"},
    "flag": {"type": "text", "analyzer": "keyword"}
  }
}
'

搜索无结果：

POST test2/_search?filter_path=hits.total
{
  "query": {
    "match": {
      "flag": "foo"
    }
  }
}

或

curl -XPOST 'localhost:9200/test2/_search?filter_path=hits.total&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "flag": "foo"
    }
  }
}
'

获取新的映射：

POST test2/_update_by_query?refresh&conflicts=proceed
POST test2、_search?filter_path=hits.total
{
  "query": {
    "match": {
      "flag": "foo"
    }
  }
}

七、Multi Get API

1、获取多文档API

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]

}

或

curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

GET /test/_mget
{
    "docs" : [
        {
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_type" : "type",
            "_id" : "2"
        }
    ]
}

或

curl -XGET 'localhost:9200/test/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

GET /test/type/_mget
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2"
        }
    ]
}

或

curl -XGET 'localhost:9200/test/type/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2"
        }
    ]
}
'

GET /test/type/_mget
{
    "ids" : ["1", "2"]
}
或

curl -XGET 'localhost:9200/test/type/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "ids" : ["1", "2"]
}
'

2、可选类型

GET /test/_mget/
{
  "docs" : [
        {
            "_type":"typeA",
            "_id" : "1"
        },
        {
            "_type":"typeB",
            "_id" : "1"
        }
    ]
}

或

curl -XGET 'localhost:9200/test/_mget/?pretty' -H 'Content-Type: application/json' -d'
{
  "docs" : [
        {
            "_type":"typeA",
            "_id" : "1"
        },
        {
            "_type":"typeB",
            "_id" : "1"
        }
    ]
}
'

3、源过滤

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_source" : false
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "_source" : ["field3", "field4"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "3",
            "_source" : {
                "include": ["user"],
                "exclude": ["user.location"]
            }
        }
    ]
}

或

curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_source" : false
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "_source" : ["field3", "field4"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "3",
            "_source" : {
                "include": ["user"],
                "exclude": ["user.location"]
            }
        }
    ]
}
'

4、域

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "stored_fields" : ["field1", "field2"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}

或

curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "stored_fields" : ["field1", "field2"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}
'

或

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "stored_fields" : ["field1", "field2"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}

或

curl -XGET 'localhost:9200/test/type/_mget?stored_fields=field1,field2&pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}
'

5、路由

GET /_mget?routing=key1
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_routing" : "key2"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}

或

curl -XGET 'localhost:9200/_mget?routing=key1&pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_routing" : "key2"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

八、批量API

1、批量操作

lxq@slave-01:~/elastic-search/elasticsearch-6.0.0/data$ cat requests
{ "index" : { "_index" : "test5", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
lxq@slave-01:~/elastic-search/elasticsearch-6.0.0/data$ sh curl.sh
{"took":131,"errors":false,"items":[{"index":{"_index":"test5","_type":"type1","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}}]}
curl.sh: 2: curl.sh: {took:7,: not found
lxq@slave-01:~/elastic-search/elasticsearch-6.0.0/data$ cat curl.sh
curl -u "elastic":"elastic" -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests";echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test5","_type":"type1","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

POST _bulk
{ "index" : { "_index" : "test4", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test4", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test4", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} }
{ "doc" : {"field2" : "value2"} }

或

curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "index" : { "_index" : "test4", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test4", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test4", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} }
{ "doc" : {"field2" : "value2"} }
'

2、版本控制

3、路由

4、等待激活分片数

5、刷新

6、更新

支持选项：  doc (partial document), upsert, doc_as_upsert, script, params (for script), 


          lang (for script) and _source

POST _bulk
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_type" : "type1", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_type" : "type1", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}

或

curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_type" : "type1", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_type" : "type1", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}'

九、重新索引API

1、本地拷贝

目的：将文档从一个索引拷贝到另一个索引

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }

}

或

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}
'

直接覆盖目标索引的同type和id的文档

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "internal"
  }
}

只覆盖低版本的目标文档

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "external"
  }
}

只补充新文档

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "op_type": "create"
  }
}

统计版本冲突次数

POST _reindex
{
  "conflicts": "proceed",
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "op_type": "create"
  }
}

拷贝query匹配的文档

POST _reindex
{
  "source": {
    "index": "twitter",
    "type": "tweet",
    "query": {
      "term": {
        "user": "kimchy"
      }
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}

多索引拷贝

POST _reindex
{
  "source": {
    "index": ["twitter", "blog"],
    "type": ["tweet", "post"]
  },
  "dest": {
    "index": "all_together"
  }
}

文档数限定拷贝

POST _reindex
{
  "size": 1,
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}

源索引域限定拷贝

POST _reindex
{
  "source": {
    "index": "twitter",
    "_source": ["user", "tweet"]
  },
  "dest": {
    "index": "new_twitter"
  }
}

脚本修改文档元信息拷贝

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "external"
  },
  "script": {
    "source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
    "lang": "painless"
  }
}

路由设定拷贝

POST _reindex
{
  "source": {
    "index": "source",
    "query": {
      "match": {
        "company": "cat"
      }
    }
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}

设置batches大小

POST _reindex
{
  "source": {
    "index": "source",
    "size": 100
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}

指定pipeline拷贝

POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
    "pipeline": "some_ingest_pipeline"
  }
}

2、远程拷贝

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

注意：如果需要用户名和密码，那么host要用https

elasticsearch.yml要将远程的ip配置到reindex.remote.whitelist

例如：

reindex.remote.whitelist： otherhost：9200, otherhost1:9200

如果远程文档大，要设置小的batch

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200"
    },
    "index": "source",
    "size": 10,
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

设置超时时间

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "socket_timeout": "1m",
      "connect_timeout": "10s"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

3、URL参数

pretty，

refresh,

wait_for_completion,

wait_for_active_shards, 


timeout,

requests_per_second.

4、Task API

获取reindex的状态

GET _tasks?detailed=true&actions=*reindex

或

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*reindex&pretty'

获取单个任务的状态

GET /_tasks/taskId:1

或

curl -XGET 'localhost:9200/_tasks/taskId:1?pretty'

取消任务

POST _tasks/task_id:1/_cancel

或

curl -XPOST 'localhost:9200/_tasks/task_id:1/_cancel?pretty'

5、rethrottling

POST _reindex/task_id:1/_rethrottle?requests_per_second=-1

或

curl -XPOST 'localhost:9200/_reindex/task_id:1/_rethrottle?requests_per_second=-1&pretty'

6、reindex并修改域名称

POST reindex/test/1?refresh
{
  "text": "words words",
  "flag": "foo"
}

或

curl -XPOST 'localhost:9200/reindex/test/1?refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "text": "words words",
  "flag": "foo"
}
'

POST _reindex
{
  "source": {
    "index": "reindex"
  },
  "dest": {
    "index": "reindex2"
  },
  "script": {
    "source": "ctx._source.tag = ctx._source.remove(\"flag\")"
  }
}

或

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "reindex"
  },
  "dest": {
    "index": "reindex2"
  },
  "script": {
    "source": "ctx._source.tag = ctx._source.remove(\"flag\")"
  }
}
'

7、手工分片

POST _reindex
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 0,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}
POST _reindex
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 1,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}

或

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 0,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}
'
curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 1,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}
'

验证

GET _refresh
POST new_twitter/_search?size=0&filter_path=hits.total

或

curl -XGET 'localhost:9200/_refresh?pretty'
curl -XPOST 'localhost:9200/new_twitter/_search?size=0&filter_path=hits.total&pretty'

8、自动切片

POST _reindex?slices=5&refresh
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}

或

curl -XPOST 'localhost:9200/_reindex?slices=5&refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}
'
验证

POST new_twitter/_search?size=0&filter_path=hits.total

或

curl -XPOST 'localhost:9200/new_twitter/_search?size=0&filter_path=hits.total&pretty'

9、picking the number of slices

10、reindex daily indices

11、extracting a random subset of an index

十、词向量

GET /bank/account/1/_termvectors

或

curl -XGET 'localhost:9200/bank/account/1/_termvectors?pretty'

GET /bank/account/1/_termvectors?fields=address

或

curl -XGET 'localhost:9200/bank/account/1/_termvectors?fields=address&pretty'

十一、多词向量API

POST /_mtermvectors
{
   "docs": [
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "2",
         "term_statistics": true
      },
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "1",
         "fields": [
            "message"
         ]
      }
   ]
}

或

curl -XPOST 'localhost:9200/_mtermvectors?pretty' -H 'Content-Type: application/json' -d'
{
   "docs": [
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "2",
         "term_statistics": true
      },
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "1",
         "fields": [
            "message"
         ]
      }
   ]
}
'

POST /twitter/_mtermvectors
{
   "docs": [
      {
         "_type": "tweet",
         "_id": "2",
         "fields": [
            "message"
         ],
         "term_statistics": true
      },
      {
         "_type": "tweet",
         "_id": "1"
      }
   ]
}