solrCloud 4.9 分布式集群部署及注意事项
来源:互联网 发布:数据整理的英语 编辑:程序博客网 时间:2024/06/04 17:50
环境搭建
一.zookeeper
参考:http://blog.chinaunix.NET/uid-25135004-id-4214399.html
现有4台机器 10.14.2.201 10.14.2.202 10.14.2.203 10.14.2.204
安装zookeeper集群
在所有机器上进行
1、下载安装包解压
tar xvf zookeeper-3.4.5.tar.gz -C /export/
cd /export/
ln -s zookeeper-3.4.5 zookeeper
2、设置环境变量
vim /etc/profile
#Set ZooKeeper Enviroment
export ZOOKEEPER_HOME=/export/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin
3、加载环境变量
source /etc/profile
4、配置zookeeper
cd /export/zookeeper/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/export/zookeeper/data
dataLogDir=/export/zookeeper/log
clientPort=2181
server.1=10.14.2.201:2888:3888
server.2=10.14.2.202:2888:3888
server.3=10.14.2.203:2888:3888
server.4=10.14.2.204:2888:3888
5、dataDir和日志目录dataLogDir
mkdir -p /export/zookeeper/data
mkdir -p /export/zookeeper/log
6、在10.14.2.201上
在dataDir目录下创建myid文件
echo 1 > /export/zookeeper/data/myid
myid中的数字与zoo.cfg中的
"server.数字=ip或者域名:port1:port2"
相对应,
配置文件中10.14.2.201对应的server.数字是1 所以myid的数字应为1
7、在10.14.2.202上
在dataDir目录下创建myid文件
echo 2 > /export/zookeeper/data/myid
8、在10.14.2.203上
在dataDir目录下创建myid文件
echo 3 > /export/zookeeper/data/myid
9、在10.14.2.204上
在dataDir目录下创建myid文件
echo 4 > /export/zookeeper/data/myid
在所用机器上执行
10、修改权限
chown hadoop.hadoop -R /export/zookeeper/
chown hadoop.hadoop -R /export/zookeeper
11、启动zookeeper
/export/zookeeper/bin/zkServer.sh start
12、观察zookeeper状态
1> 10.14.2.201
# /export/zookeeper/bin/zkServer.sh status
JMX enabled
by
default
Using config: /export/zookeeper/bin/../conf/zoo.cfg
Mode: follower
2> 10.14.2.202
# /export/zookeeper/bin/zkServer.sh status
JMX enabled
by
default
Using config: /export/zookeeper/bin/../conf/zoo.cfg
Mode: follower
3> 10.14.2.203
# /export/zookeeper/bin/zkServer.sh status
JMX enabled
by
default
Using config: /export/zookeeper/bin/../conf/zoo.cfg
Mode: leader
4> 10.14.2.204
# /export/zookeeper/bin/zkServer.sh status
JMX enabled
by
default
Using config: /export/zookeeper/bin/../conf/zoo.cfg
Mode: follower
13、此时zookeeper已安装并启动完毕
二.solrCloud
参考:http://lucien-zzy.iteye.com/blog/2002463
1.安装tomcat,配置端口。部署solr.war,将solr\example\lib\ext下的jar拷贝到solr\WEB-INF\lib下。
2.创建
/home/solr/solrcloud
/home/solr/solrcloud/config-files (放置solr\example\solr\collection1\conf 下的所有文件)
/home/solr/solrcloud/solr-lib (放置solr\WEB-INF\lib下的所有jar包)
3.配置中文分词器
4.创建solr数据目录
/home/solr/solr-cores
生成solr.xml,如下:
<?xml version="1.0" encoding="UTF-8" ?><solr persistent="true"><logging enabled="true"><watcher size="100" threshold="INFO" /></logging><cores defaultCoreName="collection1" adminPath="/admin/cores" host="${host:}" hostPort="8080" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}"></cores></solr>
5.创建
tomcat/conf/Catalina
tomcat/conf/Catalina/localhost
并创建solr.xml
<?xml version="1.0" encoding="UTF-8"?><Context docBase="/soft/apache-tomcat-7.0.37/webapps/solr" debug="0" crossContext="true"><Environment name="solr/home" type="java.lang.String" value="/home/solr/solr-cores" override="true"/></Context>
docBase根据实际情况配置;
solr/home为之前创建的目录:/home/solr/solr-cores
6.修改tomcat/bin/cataina.sh 文件,在最上方加入:JAVA_OPTS="-DzkHost=master:2181,slave1:2181,slave2:2181",或直接使用ip。
将以上配置发布到集群其他机器上。
7.SolrCloud是通过ZooKeeper集群来保证配置文件的变更及时同步到各个节点上,所以,需要将配置文件上传到ZooKeeper集群中。执行如下操作:
java -classpath .:/home/solr/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.91.128:2181,192.168.91.129:2181,192.168.91.130:2181 -confdir /home/solr/solrcloud/config-files/ -confname myconf
-cmd upconfig:上传配置文件
-confdir:配置文件的目录
-confname:指定对应的名称
java -classpath .:/home/solr/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd linkconfig -collection collection1 -confname myconf -zkhost 192.168.91.128:2181,192.168.91.129:2181,192.168.91.130:2181
-cmd linkconfig:为指定collection"绑定"配置文件
-collection
-confname
8.检查zookeeper的配置信息
zkCli.sh -server ip:port
ls /configs/myconf
ls /live_nodes
9.启动tomcat,创建collection
curl 'http://192.168.91.128:8080/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=1'
action:CREATE
name:collection名称
numShards:分片数量
replicationFactor:副本集数量
也可以通过zk查询collection的情况:ls /collections/mycollection
-------------------------------
1.ZK CLI
ZooKeeper's Command Line Utility (CLI).
zkcli
Parameters
Short
Parameter Usage
Meaning
-cmd <arg>
CLI Command to be executed: bootstrap
, upconfig
, downconfig
, linkconfig
, makepath
, get
, getfile
, put
,putfile
, list
or clear
.
This parameter is mandatory
-z
-zkhost <locations>
ZooKeeper host address.
This parameter is mandatory for all CLI commands.
-c
-collection <name>
For linkconfig
: name of the collection.
-d
-confdir <path>
For upconfig
: a directory of configuration files.
-h
-help
Display help text.
-n
-confname <arg>
For upconfig
, linkconfig
: name of the configuration set.
-r
-runzk <port>
Run ZooKeeper internally by passing the Solr run port; only for clusters on one machine.
-s
-solrhome <path>
For bootstrap
or when using -runzk
: the mandatory solrhome location.
ZooKeeper CLI Examples
Below are some examples of using the zkcli CLI:
Uploading a Configuration Directory
java -classpath example/solr-webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983-confdir example/solr/collection1/conf -confname conf1 -solrhome example/solr
Put arbitrary data into a new ZK file
java -classpath example/solr-webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -zkhost 127.0.0.1:9983 -put /data.txt 'some data'
Put a local file into a new ZK file
java -classpath example/solr-webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -zkhost 127.0.0.1:9983 -putfile /data.txt /some/local/file.txt
Linking a Collection to a Configuration Set
java -classpath example/solr-webapp/webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -cmd linkconfig -zkhost 127.0.0.1:9983-collection collection1 -confname conf1 -solrhome example/solr
Bootstrapping All the Configuration Directories in solr.xml
java -classpath example/solr-webapp/webapp/WEB-INF/lib/*org.apache.solr.cloud.ZkCLI -cmd bootstrap -zkhost 127.0.0.1:9983-solrhome example/solr
Scripts
There are scripts in example/cloud-scripts that handle the classpath and class name for you if you are using Solr out of the box with Jetty. Commands then become:
sh zkcli.sh -cmd linkconfig -zkhost 127.0.0.1:9983-collection collection1 -confname conf1 -solrhome example/solr
2.Managing collections via the collections API
CREATE---
http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=4
name:collection的名称
numShards:指定分片数量(slices)
replicationFactor:副本数量
maxShardsPerNode:默认值为1,注意三个数值:numShards、replicationFactor、liveSolrNode,一个正常的solrCloud集群不容许同一个liveSolrNode上部署同一个shard的多个replic,因此当maxShardsPerNode=1时,numShards*replicationFactor>liveSolrNode时,报错。因此正确时因满足以下条件:
numShards*replicationFactor<liveSolrNode*maxShardsPerNode
createNodeSet:
collection.configName:指定该collection使用那份config,这份config必须存在于zk中。
DELETE---
http://localhost:8983/solr/admin/collections?action=DELETE&name=mycollection
RELOAD---
http://localhost:8983/solr/admin/collections?action=RELOAD&name=mycollection
SPLIT SHARD---
http://localhost:8983/solr/admin/collections?action=SPLITSHARD&collection=<collection_name>&shard=shardId
3.Creating cores via CoreAdmin
curl 'http://localhost:8983/solr/admin/cores?action=CREATE&name=mycore&collection=collection1&shard=shard2'
-------------------------------
Q1:Multiple schemas in the same SolrCloud ?
java -classpath .:/home/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 10.200.51.190:2224,10.200.51.192:2224,10.200.51.194:2224 -confdir /home/solrcloud/fox-config-files/ -confname foxconf
指定配置文件,上传到zk。
curl 'http://10.200.51.194:2223/solr/admin/collections?action=CREATE&name=fox4&numShards=1&replicationFactor=5&collection.configName=foxconf'
为创建的collection指定collection.configName。
文章来源:http://blog.csdn.net/guochunyang/article/details/49181759
- solrCloud 4.9 分布式集群部署及注意事项
- solrCloud 4.9 分布式集群部署及注意事项
- solrCloud 4.9 分布式集群部署及注意事项
- solrCloud分布式集群部署(一)
- SOLRCLOUD分布式集群部署步骤
- solrcloud分布式集群部署&zookeeper集群安装
- SolrCloud 分布式集群安装部署以及管理
- SolrCloud 分布式集群安装部署(solr4.8.1 + zookeeper +tomcat)
- 【SOLR集群】分布式搜索部署SOLRCLOUD案例图文
- SolrCloud 分布式集群安装部署(solr+ zookeeper +tomcat)
- SolrCloud集群部署
- SolrCloud集群部署
- hadoop伪分布式集群部署注意事项
- solrCloud分布式集群安装配置
- 基于Docker 分布式部署solrCloud
- Qizmt 单机及分布式部署注意事项
- ZooKeeper分布式集群部署及问题
- 分布式搜索之搭建Solrcloud(Solr集群)
- Android点击事件分发机制
- 20170513 Win10打不开菜单且点击通知栏无反应的解决方法
- JS中的call()和apply()方法
- 动态规划考试test20170429
- Android6.0 selinux没有对某个文件的权限(又neverAllow)处理方法
- solrCloud 4.9 分布式集群部署及注意事项
- 自定义控件继承View------Music
- Java 读取.xls模板并另存为.xls
- POJ 3414 Pots(bfs+路径记录)
- 5 Best Linux Download Managers
- 什么是网络延迟
- CentOS7 64位下MySQL5.7安装与配置(YUM)
- 大学生程序设计邀请赛(华东师范大学)
- log4j配置详解