Storm之——搭建Storm集群

转载请注明出处：http://blog.csdn.net/l1028386804/article/details/78419484

一、机器规划

主机名IPZookeeperNimbusSupbervisorliuyazhuang161192.168.209.161是是否liuyazhuang162192.168.209.162是否是liuyazhuang163192.168.209.163是否是

二、搭建Java环境

1、下载JDK

可以到Oracle官网下载Linux版本的JDK，链接为：http://www.oracle.com/technetwork/java/javase/downloads/index.html 我下载的是jdk1.7.0_72

2、解压JDK

在命令行输入tar -zxvf jdk-7u72-linux-x64.tar.gz 进行解压

3、配置环境变量

在命令行输入vim /etc/profile打开profile文件，在文件末尾添加如下代码：

JAVA_HOME=/usr/local/jdk1.7.0_72  CLASS_PATH=.:$JAVA_HOME/lib  PATH=$JAVA_HOME/bin:$PATH  export JAVA_HOME CLASS_PATH PATH  

输入命令source /etc/profile使环境变量生效

4、拷贝文件

在命令行输入如下命令将JDK环境和/etc/profile文件拷贝到liuyazhuang1162和liuyazhuang163主机上。

scp -r /usr/local/jdk1.7.0_72 liuyazhuang162:/usr/localscp -r /usr/local/jdk1.7.0_72 liuyazhuang163:/usr/localscp /etc/profile liuyazhuang162:/etc/scp /etc/profile liuyazhuang163:/etc/

同时，我们分别在liuyazhuang162和liuyazhuang163主机上执行命令source /etc/profile使环境变量生效
至此，JDK环境搭建完成。

三、搭建Zookeeper集群

1、下载Zookeeper

在Apache官网下载Zookeeper,链接为：http://www.apache.org/dyn/closer.cgi/zookeeper/我下载的是zookeeper-3.4.9

2、解压Zookeeper

在命令行输入tar -zxvf zookeeper-3.4.9.tar.gz对zookeeper进行解压。

3、Zookeeper集群搭建

切换到Zookeeper的conf目录下执行以下命令

cp zoo_sample.cfg zoo.cfg 

然后输入vim zoo.cfg命令 配置如下：

# The number of milliseconds of each ticktickTime=2000# The number of ticks that the initial # synchronization phase can takeinitLimit=10# The number of ticks that can pass between # sending a request and getting an acknowledgementsyncLimit=5# the directory where the snapshot is stored.# do not use /tmp for storage, /tmp here is just # example sakes.dataDir=/usr/local/zookeeper-3.4.9/datadataLogDir=/usr/local/zookeeper-3.4.9/datalog# the port at which the clients will connectclientPort=2181# the maximum number of client connections.# increase this if you need to handle more clients#maxClientCnxns=60## Be sure to read the maintenance section of the # administrator guide before turning on autopurge.## http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance## The number of snapshots to retain in dataDir#autopurge.snapRetainCount=3# Purge task interval in hours# Set to "0" to disable auto purge feature#autopurge.purgeInterval=1server.1=liuyazhuang161:2888:3888server.2=liuyazhuang162:2888:3888server.3=liuyazhuang163:2888:3888

同时在Zookeeper目录下新建data和dataLog两个目录用来存放Zookeeper的myid文件和日志文件，

mkdir datamkdir dataLog

将目录切换到zookeeper的data目录下
执行命令

vim myid

按下键盘i输入1后，按下esc键，再按下shift+; （也就是输入冒号:），再输入wq退出，此时就在zookeeper的data目录下创建了一个myid文件，这个myid的文件中的内容为1

4、配置Zookeeper环境变量

为操作方便，我们也将Zookeeper配置到环境变量中，加上之前配置的JDK，我们在profile的配置如下：

JAVA_HOME=/usr/local/jdk1.7.0_72CLASS_PATH=.:$JAVA_HOME/libZOOKEEPER_HOME=/usr/local/zookeeper-3.4.9PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATHexport JAVA_HOME HADOOP_HOME CLASS_PATH PATH

5、拷贝文件

将Zookeeper环境和profile文件分别拷贝到liuyazhuang162和liuyazhuang163主机上。如下命令：

scp -r /usr/local/zookeeper-3.4.9 liuyazhuang162:/usr/localscp -r /usr/local/zookeeper-3.4.9 liuyazhuang163:/usr/localscp  /ect/profile liuyazhuang162:/etc/scp  /ect/profile liuyazhuang163:/etc/

同时，我们分别在liuyazhuang162和liuyazhuang163主机上执行命令source /etc/profile使环境变量生效。

6、修改其他主机的myid文件

注：别忘了将liuyazhuang162主机上Zookeeper中myid文件内容修改为2  将liuyazhuang163主机上Zookeeper中myid文件内容修改为3
至此，Zookeeper集群环境搭建完毕

四、Storm集群搭建

1、下载Storm

Apache官方下载Storm链接为：http://storm.apache.org/downloads.html  ,我这里下载的版本为apache-storm-1.1.1.tar.gz

2、解压Storm

tar -zxvf apache-storm-1.1.1.tar.gz

3、修改storm.yaml

修改storm.yaml，主要是修改如下配置

storm.zookeeper.servers:     - "192.168.209.161"     - "192.168.209.162"     - "192.168.209.163"storm.zookeeper.port: 2181storm.local.dir: "/usr/local/apache-storm-1.1.1/data"# nimbus.seeds: ["192.168.209.161"]supervisor.slots.ports:    - 6700    - 6701    - 6702    - 6703

其中storm.zookeeper.servers，表示Zookeeper集群地址。如果Zookeeper集群使用的不是默认端口，那么还需要配置storm.zookeeper.port。storm.local.dir用于配置Storm存储少量文件的路径。nimbus.seeds用于配置主控节点的地址，可以配置多个。

4、拷贝Storm

scp -r /usr/local/apache-storm-1.1.1/ liuyazhuang162:/usr/local/scp -r /usr/local/apache-storm-1.1.1/ liuyazhuang163:/usr/local/

5、配置环境变量并拷贝

vim /etc/profileJAVA_HOME=/usr/local/jdk1.7.0_72CLASS_PATH=.:$JAVA_HOME/libSTORM_HOME=/usr/local/apache-storm-1.1.1ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.9PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATHexport JAVA_HOME HADOOP_HOME CLASS_PATH PATH STORM_HOMEscp  /ect/profile liuyazhuang162:/etc/scp  /ect/profile liuyazhuang163:/etc/

同时，我们分别在liuyazhuang162和liuyazhuang163主机上执行命令source /etc/profile使环境变量生效
6、启动Storm

192.168.209.161 后台运行 nimbus

bin/storm nimbus >/dev/null 2>&1 &

192.168.209.162 192.168.209.163后台运行 supervisor

bin/storm supervisor >/dev/null 2>&1 &

192.168.209.161 后台运行 storm ui
storm ui 得要在 nimbus 机子上运行，不能够在 supervisor 机子上运行

bin/storm ui >/dev/null 2>&1 &

五、访问Storm UI

在浏览器中输入链接http://192.168.209.161:8080，如下图所示：

六、问题及解决方案

1、Failed to Sync Supervisor

16373 [Thread-10] ERROR o.a.s.d.s.ReadClusterState - Failed to Sync Supervisorjava.lang.RuntimeException: java.lang.InterruptedException        at org.apache.storm.utils.Utils.wrapInRuntime(Utils.java:1531) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.zookeeper.zookeeper.getChildren(zookeeper.java:265) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.cluster.ZKStateStorage.get_children(ZKStateStorage.java:174) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.cluster.StormClusterStateImpl.assignments(StormClusterStateImpl.java:153) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.daemon.supervisor.ReadClusterState.run(ReadClusterState.java:126) [storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.event.EventManagerImp$1.run(EventManagerImp.java:54) [storm-core-1.1.1.jar:1.1.1]Caused by: java.lang.InterruptedException        at java.lang.Object.wait(Native Method) ~[?:1.8.0_91]        at java.lang.Object.wait(Object.java:502) ~[?:1.8.0_91]        at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1588) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1625) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:226) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:219) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:216) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:207) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:40) ~[storm-core-1.1.1.jar:1.1.1]        at org.apache.storm.zookeeper.zookeeper.getChildren(zookeeper.java:260) ~[storm-core-1.1.1.jar:1.1.1]        ... 4 more

部署的 storm 服务器版本为 1.1.1，而提交给 storm 跑的 jar 包为：storm-starter-1.0.2.jar。版本不对应导致上面的问题。应该是 storm-core-1.1.1.jar 的代码与 storm-core-1.0.2.jar 相差较大，或者修改了通信的协议导致。将 storm 服务器版本修改为 1.0.0 即可解决问题

2、有多个 supervisor 但 storm ui 上只显示一个

具体现象就是启动了多个 supervisor，单在 ui 上只显示一个（也有可能是多个 supervisor 中的某几个看上去被“合并”了），kill 掉其中任意一个 supervisor，另一个就出现。
例如本例中有两个 supervisor，192.168.209.162 和 192.168.209.163，但是通过接口请求，每次都只会显示其中的一个，但是现实的机子是交替出现的：

解决方案：storm.yaml 文件中有配置 storm.local.dir: "/usr/local/apache-storm-1.1.1/data"，local.dir 所指目录，重启即可解决问题。原因是由于部署时通过 linux scp 命令直接分发软件到其他机子，残留了 local.dir 的东西，而 storm 是根据 local.dir 中的某一个或一些文件计算出一个 supervisor id 的。删除 local.dir 后，会重新生成 id。

3、Could not find or load main class org.apache.storm.starter.ExclamationTopology

bin/storm jar /usr/local/apache-storm-1.1.1/storm-starter-1.0.2.jar org.apache.storm.starter.ExclamationTopology et
第一确保 storm-starter-1.0.2.jar 的路径是正确的；第二保证 packagename.ExclamationTopology，包名packagename 与 类名 ExclamationTopology 是正确的