Storm-0.10.0集群安装

一、环境

两台虚拟机，ubuntu-14.04.3

二、关闭防火墙，配置hosts

root@ubuntu:~# cat /etc/hosts127.0.0.1localhost127.0.1.1ubuntu192.168.254.130 storm1192.168.254.131 storm2

三、安装Java（JDK 6+），并配置环境变量

root@ubuntu:~# cat /etc/profileexport JAVA_HOME=/usr/local/jdk1.7.0_80  export JRE_HOME=/$JAVA_HOME/jreexport PATH=$JAVA_HOME/bin:$PATHexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

四、安装Python（2.6.6+）

确定系统自带的Python版本，如果是2.6.6+可以不用再安装。

root@ubuntu:~# python -VPython 2.7.6

五、搭建ZooKeeper集群

root@ubuntu:/usr/local# wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.7/zookeeper-3.4.7.tar.gz

root@ubuntu:/usr/local# tar -zxvf zookeeper-3.4.7.tar.gzroot@ubuntu:/usr/local# cd zookeeper-3.4.7/confroot@ubuntu:/usr/local/zookeeper-3.4.7/conf# cp -p zoo_sample.cfg zoo.cfgroot@ubuntu:/usr/local/zookeeper-3.4.7/conf# vim zoo.cfg# The number of milliseconds of each ticktickTime=2000# The number of ticks that the initial # synchronization phase can takeinitLimit=10# The number of ticks that can pass between # sending a request and getting an acknowledgementsyncLimit=5# the directory where the snapshot is stored.# do not use /tmp for storage, /tmp here is just # example sakes.dataDir=/opt/zookeeper/data# the port at which the clients will connectclientPort=2181server.1=storm1:7000:7001server.2=storm2:7000:7001# the maximum number of client connections.# increase this if you need to handle more clients#maxClientCnxns=60## Be sure to read the maintenance section of the # administrator guide before turning on autopurge.## http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance## The number of snapshots to retain in dataDir#autopurge.snapRetainCount=3# Purge task interval in hours# Set to "0" to disable auto purge feature#autopurge.purgeInterval=1

# 手动创建dataDir目录root@ubuntu:/opt# mkdir zookeeperroot@ubuntu:/opt# cd zookeeperroot@ubuntu:/opt/zookeeper# mkdir data

# 在dataDir目录下创建myid文件，写id号，用来标识当前主机root@ubuntu:/opt/zookeeper/data# echo "1" > myid

storm2节点重复上面操作，也可以直接复制，只是myid输入的值是2。

# 启动ZooKeeperroot@storm1:/usr/local/zookeeper-3.4.7# bin/zkServer.sh startroot@storm2:/usr/local/zookeeper-3.4.7# bin/zkServer.sh start# 查看ZooKeeper状态root@storm1:/usr/local/zookeeper-3.4.7# bin/zkServer.sh statusZooKeeper JMX enabled by defaultUsing config: /usr/local/zookeeper-3.4.7/bin/../conf/zoo.cfgMode: followerroot@storm1:/usr/local/zookeeper-3.4.7# bin/zkServer.sh statusZooKeeper JMX enabled by defaultUsing config: /usr/local/zookeeper-3.4.7/bin/../conf/zoo.cfgMode: leader

六、安装Storm

# 下载安装Stormroot@ubuntu:/usr/local# wget wget http://mirrors.cnnic.cn/apache/storm/apache-storm-0.10.0/apache-storm-0.10.0.tar.gzroot@ubuntu:/usr/local# tar -zxvf apache-storm-0.10.0.tar.gzroot@ubuntu:/usr/local# mv apache-storm-0.10.0 storm-0.10.0

# 省略...########### These MUST be filled in for a storm configuration storm.zookeeper.servers:     - "storm1"     - "storm2"#  nimbus.host: "storm1" storm.local.dir: "/opt/storm/data" supervisor.slots.port:     - 6700     - 6701     - 6702     - 6703 nimbus.childopts: "-Xmx1024m" ui.childopts: "-Xmx768m"# # ##### These may optionally be filled in:#    ## List of custom serializations# topology.kryo.register:#     - org.mycompany.MyType#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer### List of custom kryo decorators# topology.kryo.decorators:#     - org.mycompany.MyDecorator### Locations of the drpc servers# drpc.servers:#     - "server1"#     - "server2"## Metrics Consumers# topology.metrics.consumer.register:#   - class: "backtype.storm.metric.LoggingMetricsConsumer"#     parallelism.hint: 1#   - class: "org.mycompany.MyMetricsConsumer"#     parallelism.hint: 1#     argument:#       - endpoint: "metrics-collector.mycompany.org"

# 创建storm.local.dir目录root@ubuntu:/opt# mkdir stormroot@ubuntu:/opt/storm# mkdir dataroot@ubuntu:/opt/storm/data# pwd/opt/storm/data

storm2节点重复上面操作，共同部分可以直接复制。

# 启动Storm（确保ZooKeeper已正常启动）root@storm1:/usr/local/storm-0.10.0# bin/storm nimbus >/dev/null 2>&1 &root@storm2:/usr/local/storm-0.10.0# bin/storm supervisor >/dev/null 2>&1 &root@storm1:/usr/local/storm-0.10.0# bin/storm ui >/dev/null 2>&1 &

# 查看启动进程root@storm1:~# jps2658 QuorumPeerMain2696 nimbus2813 core3334 Jpsroot@ubuntu:~# jps2673 supervisor3287 Jps2632 QuorumPeerMain

七、遇到的问题

如果出现IPv6引起的连接问题可以修改Storm启动JVM参数，如下：

 nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true"
 ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true"