Elasticsearch5.4集群（一）安装部署

ES版本升级

生产环境用的是1.7，Elasticsearch5.x在性能上有了很大的提升，计划升级到5.4，先在线下部署验证，过程中发现很多配置项都改了，各种报错。
一定要看官方文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html，网上找到的很多都过时的低版本配置，要么不对要么不全，还需要注意下breaking changes(不兼容变更项)。

节点功能分离

master、data、client分开部署，规划存储容量、内存、CPU(权威指南：https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html)。master节点不存储数据，内存和cpu都比较低，一般部署3个master保证可靠。不建议用一块很大的磁盘，建议用多块SSD，可以并发IO。不是clien的节点可以不开启http服务的节点(http.enabled: false)同时也不要安装head, bigdesk, marvel等监控插件，这样保证data节点服务器只需处理创建/更新/删除/查询索引数据等操作。

内存分配

内存不要超过32G，java版本使用jdk1.8(We recommend installing Java version 1.8.0_131 or later)，es最大分配-Xms32766m -Xmx32766m（内存对象指针压缩技术），GC使用G1不要设置-Xmn(参照垃圾回收配置：http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html)。物理机上的内存之分一半给ES，另一半给Lucene，否则性能会受影响。128G内存的机器最多部署2个es实例。

系统配置修改

禁用swap（在一些内核版本，swappness=0会引发OOM）。vim /etc/sysctl.conf
修改 vm.swappiness=1，设置vm.max_map_count=262144(否则启动会报错：max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144])

vm.swappiness=1vm.max_map_count=262144

立即生效需执行 sysctl -p
修改es用户限制，不能用root启动Elasticsearch，比如使用tomcat用户（用户重新登录生效）
vim /etc/security/limits.conf

tomcat soft memlock unlimitedtomcat hard memlock unlimited

安装Elasticsearch5.4.1

安装步骤参照（https://www.elastic.co/downloads/elasticsearch），我们把es安装到/usr/local/目录下

cd /usr/local/wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.1.tar.gzsha1sum elasticsearch-5.4.1.tar.gz tar -xzf elasticsearch-5.4.1.tar.gzcd elasticsearch-5.4.1/

建立软链接:

ln -s /usr/local/elasticsearch-5.4.1 elasticsearch

更新安装目录/usr/local/elasticsearch/bin/elasticsearch启动脚本，修改查找jvm配置文件顺序，加入"$LEIDA_ES_HOME"/config/jvm.options，让虚拟机参数优先使用实例自己的配置，因为我们一个物理机上会启用多个es实例

if [ -z "$ES_JVM_OPTIONS" ]; then    for jvm_options in "$LEIDA_ES_HOME"/config/jvm.options \                   "$ES_HOME"/config/jvm.options \                      /etc/elasticsearch/jvm.options; do        if [ -r "$jvm_options" ]; then            ES_JVM_OPTIONS=$jvm_options            break        fi    donefi

修改JVM参数的堆大小和G1GC方式(vim /usr/local/elasticsearch/config/jvm.options):

-Xms32766m-Xmx32766m## GC configuration#-XX:+UseConcMarkSweepGC#-XX:CMSInitiatingOccupancyFraction=75#-XX:+UseCMSInitiatingOccupancyOnly## G1GC-XX:+UseG1GC-XX:MaxGCPauseMillis=800-XX:ParallelGCThreads=15-XX:ConcGCThreads=4

mmseg分词器

下载elasticsearch-analysis-mmseg-5.4.1.zip
下载地址：https://github.com/medcl/elasticsearch-analysis-mmseg/releases
(elasticsearch.yml不支持参数path.plugins参数，如果一台机器上启多个实例，想配置每个实例自己的插件，比如master节点不加载head插件，需要改下源码读取路径和check验证，默认是读取安装路径下的/usr/local/elasticsearch/plugins/)

cd /usr/local/elasticsearch/plugins/unzip elasticsearch-analysis-mmseg-5.4.1.zip -d elasticsearch-analysis-mmseg-5.4.1

创建master

实例配置存放在/data/es/es_[tcpport]下

cd /data/esmkdir -p es_9300/binmkdir -p es_9300/configmkdir -p es_9300/datamkdir -p es_9300/logscp /usr/local/elasticsearch/config/log4j2.properties /data/es/es_9300/config/log4j2.propertiescp /usr/local/elasticsearch/config/jvm.options /data/es/es_9300/config/jvm.options

修改JVM堆大小，vim /data/es/es_9300/config/jvm.options，安装目录的默认-Xms32766m是给datanode用的，master的堆需要改小。

cp /usr/local/elasticsearch/config/elasticsearch.yml /data/es/es_9300/config/elasticsearch.yml
修改elasticsearch.yml如下, IP、端口和存储路径需要自己替换，其他2个master的配置类似。为避免脑裂，discovery.zen.minimum_master_nodes设为2（非生产环境验证优先保证可用这里设为1）

bootstrap.memory_lock: truebootstrap.system_call_filter: falsecluster.name: loganalysisnode.attr.box_type: masternode.name: master_1node.master: truenode.data: falsediscovery.zen.minimum_master_nodes: 1discovery.zen.ping.unicast.hosts: ["master1ip:9300","master2ip:9300","master3ip:9300"]discovery.zen.fd.ping_timeout: 50sdiscovery.zen.fd.ping_retries: 6cluster.routing.allocation.node_initial_primaries_recoveries: 6cluster.routing.allocation.node_concurrent_recoveries: 6cluster.routing.allocation.cluster_concurrent_rebalance: 4network.host: master1iphttp.port: 9200transport.tcp.port: 9300path.conf: /data/es/es_9300/configpath.data: /data/es/es_9300/datapath.logs: /data/es/es_9300/logshttp.cors.enabled: truehttp.cors.allow-origin: /.*/http.cors.allow-credentials: true

启动脚本

因为一个物理机上只安装了一个es，但是要启用多个es实例，需要区分config文件的路径，把别的脚本拷贝过来改成我们的leidaES.sh，启动master

cd /data/es/es_9300/bin./leidaES.sh startelasticsearch's real path : /usr/local/elasticsearch-5.4.1Starting elasticsearch es_9300:                            [  OK  ]

脚本内容其实就是加了些验证和start、stop参数，停止命令./leidaES.sh stop

#!/bin/sh# # init.d / servicectl compatibility (openSUSE)#if [ -f /etc/rc.status ]; then    . /etc/rc.status    rc_resetfi## Source function library.#if [ -f /etc/rc.d/init.d/functions ]; then    . /etc/rc.d/init.d/functionsfiMAX_OPEN_FILES=600000# max locked memory   (kbytes, -l) 32G=1024*1024*32MAX_LOCKED_MEMORY=33554432# vim /etc/sysctl.conf add vm.max_map_count=262144#MAX_MAP_COUNT=262144ES_USER="tomcat"ES_GROUP="tomcat"JAVA_HOME=/usr/local/java/jdk1.8ES_HOME="/usr/local/elasticsearch"SCRIPT="$0"# determine leida elasticsearch homeLEIDA_ES_HOME=`dirname "$SCRIPT"`/..# make LEIDA ELASTICSEARCH_HOME absoluteLEIDA_ES_HOME=`cd "$LEIDA_ES_HOME"; pwd`ES_NODE=`echo $LEIDA_ES_HOME|awk -F'/' '{print $NF}'`CONF_DIR="${LEIDA_ES_HOME}/config"WORK_DIR="/tmp/elasticsearch"CONF_FILE="${CONF_DIR}/elasticsearch.yml"#export ES_HOME#export JAVA_HOMEexport LEIDA_ES_HOMEexec="$ES_HOME/bin/elasticsearch"if [ -f "$exec" ]; then    chmod 755 "$exec"fiprog="elasticsearch"pidfile="$LEIDA_ES_HOME/${prog}.pid"lockfile=/var/lock/subsys/$prog# backwards compatibility for old config sysconfig files, pre 0.90.1if [ -n $USER ] && [ -z $ES_USER ] ; then   ES_USER=$USERficheckJava() {    if [ -x "$JAVA_HOME/bin/java" ]; then        JAVA="$JAVA_HOME/bin/java"    else        JAVA=`which java`    fi    if [ ! -x "$JAVA" ]; then        echo "Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME"        exit 1    fi}start() {    checkJava    [ -x $exec ] || echo "$exec script file is not executable." ||exit 5    [ -f $CONF_FILE ] || echo "$CONF_FILE is not exists." || exit 6#    if [ -n "$MAX_LOCKED_MEMORY" -a -z "$ES_HEAP_SIZE" ]; then#        echo "MAX_LOCKED_MEMORY is set - ES_HEAP_SIZE must also be set"#        return 7#    fi    if [ -n "$MAX_OPEN_FILES" ]; then        ulimit -n $MAX_OPEN_FILES    fi    if [ -n "$MAX_LOCKED_MEMORY" ]; then        ulimit -l $MAX_LOCKED_MEMORY    fi    if [ -n "$MAX_MAP_COUNT" -a -f /proc/sys/vm/max_map_count ]; then        sysctl -q -w vm.max_map_count=$MAX_MAP_COUNT    fi    if [ -n "$WORK_DIR" ]; then        mkdir -p "$WORK_DIR"        chown "$ES_USER":"$ES_GROUP" "$WORK_DIR"    fi    # Ensure that the LEIDA_ES_HOME exists (it is cleaned at OS startup time)    if [ -n "$LEIDA_ES_HOME" ] && [ ! -e "$LEIDA_ES_HOME" ]; then        mkdir -p "$LEIDA_ES_HOME" && chown "$ES_USER":"$ES_GROUP" "$LEIDA_ES_HOME"    fi    chown -R "$ES_USER":"$ES_GROUP" "$LEIDA_ES_HOME"    chown -R "$ES_USER":"$ES_GROUP" "${ES_HOME}"if [ -h "$ES_HOME" ]; then        eslink=`ls -ld "$ES_HOME"`        # Drop everything prior to ->        eslink=`expr "$eslink" : '.*-> \(.*\)$'`    echo "elasticsearch's real path : $eslink"        if expr "$eslink" : '/.*' > /dev/null; then            chown -R "$ES_USER":"$ES_GROUP" "${eslink}"        fi    fi    if [ -n "$pidfile" ] && [ ! -e "$pidfile" ]; then        touch "$pidfile" && chown "$ES_USER":"$ES_GROUP" "$pidfile"    fi    echo -n $"Starting $prog ${ES_NODE}: "    # if not running, start it up here, usually something like "daemon $exec" # -Djava.io.tmpdir=${WORK_DIR}    daemon --user $ES_USER --pidfile $pidfile $exec -d -p $pidfile -Epath.conf=${CONF_DIR}    retval=$?    echo    [ $retval -eq 0 ] && touch $lockfile    return $retval}stop() {    echo -n $"Stopping $prog ${ES_NODE}: "    # stop it here, often "killproc $prog"    killproc -p $pidfile -d 20 $prog    retval=$?    echo    [ $retval -eq 0 ] && rm -f $lockfile    return $retval}restart() {    stop    start}reload() {    restart}force_reload() {    restart}rh_status() {    # run checks to determine if the service is running or use generic status    status -p $pidfile $prog}rh_status_q() {    rh_status >/dev/null 2>&1}case "$1" in    start)        rh_status_q && exit 0        $1        ;;    stop)        rh_status_q || exit 0        $1        ;;    restart)        $1        ;;    reload)        rh_status_q || exit 7        $1        ;;    force-reload)        force_reload        ;;    status)        rh_status        ;;    condrestart|try-restart)        rh_status_q || exit 0        restart        ;;    *)        echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"        exit 2esacexit $?

创建datanode

和master节点的配置类似，注意node.master:false和node.data:true；node.name在集群中唯一。生产环境我们要做冷热分离，es1.7是通过分组来实现，ES5已改成node.attr.box_type。另外最好datanode和clientnode独立，非生产环境就合二为一了。cp /usr/local/elasticsearch/config/elasticsearch.yml /data/es/es_9301/config/elasticsearch.yml修改elasticsearch.yml如下：

bootstrap.memory_lock: truebootstrap.system_call_filter: falsecluster.name: loganalysisnode.attr.box_type: hotnode.name: hot_1node.master: falsenode.data: truediscovery.zen.minimum_master_nodes: 1discovery.zen.ping.unicast.hosts: ["master1ip:9300","master2ip:9300","master3ip:9300"]discovery.zen.fd.ping_timeout: 50sdiscovery.zen.fd.ping_retries: 6cluster.routing.allocation.node_initial_primaries_recoveries: 6cluster.routing.allocation.node_concurrent_recoveries: 6cluster.routing.allocation.cluster_concurrent_rebalance: 4network.host: 改成机器的IPhttp.port: 9201transport.tcp.port: 9301path.conf: /data/es/es_9301/configpath.data: /data/es/es_9301/datapath.logs: /data/es/es_9301/logshttp.cors.enabled: truehttp.cors.allow-origin: /.*/http.cors.allow-credentials: true

启动数据节点

先启动3个master节点，再启动配置的多个数据节点，组成Es集群

cd /data/es/es_9301/bin./leidaES.sh startelasticsearch's real path : /usr/local/elasticsearch-5.4.1Starting elasticsearch es_9301:                            [  OK  ]

创建动态模版

配置mmseg分词和默认一个分片，不要副本。每个doc都有time字段和存储日志的msg字段。

PUT /_template/default_template{   "template": "*",   "settings": {      "index.refresh_interval": "15s",      "index.translog.flush_threshold_size": "1gb",  "index.translog.sync_interval": "1000000ms",  "index.codec": "best_compression",      "index.number_of_replicas": "0",      "index.number_of_shards": "1"   },   "mappings": {      "_default_": {         "_source": {         },         "dynamic_templates": [            {               "msg": {                  "path_match": "msg",                  "mapping": {                     "analyzer": "mmseg_maxword",                     "search_analyzer": "mmseg_maxword",                     "omit_norms": true,                     "type": "text"                  }               }            },            {               "time": {                  "path_match": "time",                  "mapping": {                     "type": "date"                  }               }            },            {               "other": {                  "mapping": {                     "fielddata": {                        "format": "doc_values"                     },                     "index": "not_analyzed",                     "ignore_above":1000,                     "omit_norms": true,                     "doc_values": true                  },                  "match": "*"               }            }         ],         "_all": {            "enabled": false         }      }   },   "aliases": {}}

测试中文分词`mmseg_maxword` ,`mmseg_complex` ,`mmseg_simple`

curl -X GET "localhost:9200/_analyze?analyzer=mmseg_maxword&pretty=true" -d '中华人民共和国12**3朝阳群众，www.elastic.co'

集群状态API接口测试

查看集群状态
curl 'localhost:9200/_cat/health?v'

创建索引（sense插件执行）
PUT /test

插入doc数据
POST /test/china/1
{"msg":"美国留给伊拉克的是个烂摊子吗"}
POST /test/china/2
{"msg":"公安部：各地校车将享最高路权"}
POST /test/china/3
{"msg":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
POST /test/china/4
{"msg":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}

搜索数据，能搜到2条数据。
POST /test/china/_search?
{"query":{"bool":{"must":[{"match_phrase_prefix":{"msg":{"query":"中国"}}}],"must_not":[]}},"from":0,"size":40}

至此集群正常运行，还需安装监控插件，后续优化配置