Storm安装介绍
来源：互联网 发布：零售行业有哪些数据 编辑：程序博客网 时间：2024/06/06 04:40
将安装过storm 0.9.1的过程流水记录，稍微整理了一下，以备日后查看。
0. introduction配置项说明： storm在conf/storm.yaml包含了一些配置信息。 storm.zookeeper.servers/port：配置storm集群使用的zookeeper集群的地址/端口 storm.local.dir：Nimbus和Supervisor在本地磁盘上cached状态信息(jar包和配置文件等) nimbus.host：Nimbus机器的地址 ui.port： 用于storm ui的端口 supervisor.slots.ports：机器上运行的进程及每个进程使用的端口 storm UI 默认端口是8080，当该端口已被占用时，需要在storm.yaml中显式指定启动storm说明： storm是快速失效（fail-fast）系统，有错误的时候它都会退出的。如果Nimbus和Supervisor重启的话，不会影响正在运行的topology。 zookeeper：在zookeeper的机器上启动进程，/usr/local/zookeeper/bin/zkServer.sh start Nimbus：在Nimbus机器上启动进程： storm nimbus Supervisor：在每台Supervisor机器上启动进程：storm supervisor Storm UI：storm UI是一个监控storm运行状态的页面：storm ui
页面访问地址: http://<u>nimbus.host</u>:<u>ui.port </u> 可写一个简单的脚本来启动/关闭storm： 启动前： kill storm相关进程：ps aux|grep storm|grep root|awk '{print $2}'|xargs sudo kill >>> 如果有“kill xxxx: 没有那个进程”，删除storm.local.dir中的supervisor缓存文件，再重新启动。这个问题可能是由于之前的Supervisor的异常终止导致 后台启动进程：nohup storm nimbus & 编译 使用maven编译 编译：mvn compile 打包：mvn package 删除：mvn clean 运行 向storm集群提交storm_starter中的example 提交topology：sudo storm jar storm-starter-*.jar storm.starter.WordCountTopology WordCount 终止topology：storm kill WordCount－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－1. install javaGoogle for it... 2. install python 2.7Google for it too...3. install zookeeper (a)建立zookeeper snapshot的目录： [irobot@TEST storm]$ mkdir zksnapshot [irobot@TEST storm]$ ls apache-storm-0.9.1-incubating apache-storm-0.9.1-incubating.zip zksnapshot zookeeper-3.4.6 zookeeper-3.4.6.tar.gz [irobot@TEST storm]$ cd zksnapshot/ [irobot@TEST zksnapshot]$ pwd /home/irobot/storm/zksnapshot (b)下载zookeeper： [irobot@TEST storm]$ wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz [irobot@TEST storm]$ ls apache-storm-0.9.1-incubating apache-storm-0.9.1-incubating.zip zookeeper-3.4.6 zookeeper-3.4.6.tar.gz [irobot@TEST storm]$ pwd /home/irobot/storm [irobot@TEST storm]$ cd zookeeper-3.4.6 [irobot@TEST zookeeper-3.4.6]$ ls bin CHANGES.txt contrib docs ivy.xml LICENSE.txt README_packaging.txt recipes zookeeper-3.4.6.jar zookeeper-3.4.6.jar.md5 build.xml conf dist-maven ivysettings.xml lib NOTICE.txt README.txt src zookeeper-3.4.6.jar.asc zookeeper-3.4.6.jar.sha1 [irobot@TEST zookeeper-3.4.6]$ cd conf/ [irobot@TEST conf]$ ls configuration.xsl log4j.properties zoo_sample.cfg [irobot@TEST conf]$ cp zoo_sample.cfg zoo.cfg [irobot@TEST conf]$ more zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. #dataDir=/tmp/zookeeper <=== 更改snapshot路径 dataDir=/home/irobot/storm/zksnapshot # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature autopurge.purgeInterval=1 [irobot@TEST conf]$ [irobot@TEST conf]$ ls configuration.xsl log4j.properties zoo.cfg zoo_sample.cfg [irobot@TEST conf]$ (c)设置zookeeper环境变量 [irobot@TEST zookeeper-3.4.6]$ ls bin CHANGES.txt contrib docs ivy.xml LICENSE.txt README_packaging.txt recipes zookeeper-3.4.6.jar zookeeper-3.4.6.jar.md5 build.xml conf dist-maven ivysettings.xml lib NOTICE.txt README.txt src zookeeper-3.4.6.jar.asc zookeeper-3.4.6.jar.sha1 [irobot@TEST zookeeper-3.4.6]$ pwd /home/irobot/storm/zookeeper-3.4.6 [irobot@TEST zookeeper-3.4.6]$ export ZOOKEEPER_HOME=/home/irobot/storm/zookeeper-3.4.6 [irobot@TEST zookeeper-3.4.6]$ export PATH=$ZOOKEEPER_HOME/bin:$PATH 简单方式，将环境变量设置：直接放入 .bash_profile export ZOOKEEPER_HOME=/home/irobot/storm/zookeeper-3.4.6 export PATH=$ZOOKEEPER_HOME/bin:$PATH (d)启动服务器，并用客户端连上去测试： [irobot@TEST zookeeper-3.4.6]$ bin/zkServer.sh start [irobot@TEST zookeeper-3.4.6]$ bin/zkCli.sh -server 127.0.0.1:21814. install storm 0.9.1 (a)下载 storm 0.9.1 下载地址： https://github.com/apache/storm/archive/apache-0.9.1.zip (b)解压 [irobot@TEST storm]$ ls -al total 18344 drwxr-xr-x 5 sms oinstall 4096 May 25 15:36 . drwx------. 50 sms oinstall 4096 May 25 15:38 .. drwxr-xr-x 8 sms oinstall 4096 May 25 15:22 apache-storm-0.9.1-incubating -rw-r--r-- 1 sms oinstall 1060431 May 25 15:22 apache-storm-0.9.1-incubating.zip drwxr-xr-x 3 sms oinstall 4096 May 25 15:43 zksnapshot drwxr-xr-x 10 sms oinstall 4096 May 25 15:43 zookeeper-3.4.6 -rw-r--r-- 1 sms oinstall 17699306 Oct 31 2014 zookeeper-3.4.6.tar.gz [irobot@TEST storm]$ tar xzvf apache-storm-0.9.1-incubating [irobot@TEST storm]$ ls -al apache-storm-0.9.1-incubating total 148 drwxr-xr-x 8 sms oinstall 4096 May 25 15:22 . drwxr-xr-x 5 sms oinstall 4096 May 25 15:36 .. drwxr-xr-x 2 sms oinstall 4096 Jan 12 2014 bin -rw-r--r-- 1 sms oinstall 29410 Jan 12 2014 CHANGELOG.md drwxr-xr-x 2 sms oinstall 4096 Jan 12 2014 conf -rw-r--r-- 1 sms oinstall 295 Jan 12 2014 .gitignore -rw-r--r-- 1 sms oinstall 4794 Jan 12 2014 KEYS -rw-r--r-- 1 sms oinstall 15255 Jan 12 2014 LICENSE drwxr-xr-x 2 sms oinstall 4096 Jan 12 2014 logback -rw-r--r-- 1 sms oinstall 382 Jan 12 2014 NOTICE -rw-r--r-- 1 sms oinstall 25465 Jan 12 2014 pom.xml -rw-r--r-- 1 sms oinstall 7262 Jan 12 2014 README.markdown drwxr-xr-x 4 sms oinstall 4096 Jan 12 2014 storm-core drwxr-xr-x 3 sms oinstall 4096 Jan 12 2014 storm-deps drwxr-xr-x 3 sms oinstall 4096 Jan 12 2014 storm-dist -rw-r--r-- 1 sms oinstall 9959 Jan 12 2014 TODO -rw-r--r-- 1 sms oinstall 19 Jan 12 2014 VERSION (c) 配置storm 以下过程是基于单机模式。如果是集群模式，Nimbus和ui只在一台机器上启动；其余机器上启动supervisor和logview。 [irobot@TEST apache-storm-0.9.1-incubating]$ pwd /home/irobot/storm/apache-storm-0.9.1-incubating [irobot@TEST apache-storm-0.9.1-incubating]$ ls bin CHANGELOG.md conf KEYS LICENSE logback NOTICE pom.xml README.markdown storm-core storm-deps storm-dist TODO VERSION [irobot@TEST apache-storm-0.9.1-incubating]$ cd conf/ [irobot@TEST conf]$ ls defaults.yaml jaas_digest.conf logback.xml storm.yaml.example [irobot@TEST conf]$ cp storm.yaml.example storm.yaml [irobot@TEST conf]$ ls defaults.yaml jaas_digest.conf logback.xml storm.yaml storm.yaml.example [irobot@TEST conf]$ [irobot@TEST conf]$ cat storm.yaml[irobot@TEST conf]$ cat storm.yaml# Licensed to the Apache Software Foundation (ASF) under one# or more contributor license agreements. See the NOTICE file# distributed with this work for additional information# regarding copyright ownership. The ASF licenses this file# to you under the Apache License, Version 2.0 (the# "License"); you may not use this file except in compliance# with the License. You may obtain a copy of the License at## http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.########### These MUST be filled in for a storm configuration# storm.zookeeper.servers:# - "server1"# - "server2"## nimbus.host: "nimbus"storm.zookeeper.servers: - "192.168.1.100"storm.zookeeper.port: 2181storm.local.dir: "/home/irobot/storm/stormlocaldir"nimbus.host: "192.168.1.100"ui.port: 8088storm.messaging.transport: "backtype.storm.messaging.netty.Context"storm.messaging.netty.server_worker_threads: 1storm.messaging.netty.client_worker_threads: 1storm.messaging.netty.buffer_size: 5242880storm.messaging.netty.max_retries: 100storm.messaging.netty.max_wait_ms: 1000storm.messaging.netty.min_wait_ms: 100supervisor.slots.ports: - 6700 - 6701 - 6702 - 6703### ##### These may optionally be filled in:### List of custom serializations# topology.kryo.register:# - org.mycompany.MyType# - org.mycompany.MyType2: org.mycompany.MyType2Serializer### List of custom kryo decorators# topology.kryo.decorators:# - org.mycompany.MyDecorator### Locations of the drpc servers# drpc.servers:# - "server1"# - "server2"## Metrics Consumers# topology.metrics.consumer.register:# - class: "backtype.storm.metrics.LoggingMetricsConsumer"# parallelism.hint: 1# - class: "org.mycompany.MyMetricsConsumer"# parallelism.hint: 1# argument:# - endpoint: "metrics-collector.mycompany.org" (d) 启动 storm后台服务 环境变量设置：直接放入 .bash_profile export STORM_HOME=/home/irobot/storm/apache-storm-0.9.1-incubating export PATH=$PATH:$STORM_HOME/bin （可选）先杀掉之前存在的storm后台服务： [irobot@TEST apache-storm-0.9.1-incubating]$ ps aux | grep storm |awk '{print $2}'|xargs kill 启动后台服务： [irobot@TEST apache-storm-0.9.1-incubating]$ storm nimbus >/dev/null 2>&1 & [1] 2888 可能存在异常： 主要是因为正式的版本中有个RELEASE文件。 ****************************************** The storm client can only be run from within a release. You appear to be trying to run the client from a checkout of Storm's source code. You can download a Storm release at http://storm-project.net/downloads.html ****************************************** 解决方式： 从http://storm-project.net/downloads.html 重新下载 正式的storm版本。 [irobot@TEST apache-storm-0.9.1-incubating]$ storm supervisor>/dev/null 2>&1 & [2] 2979 [irobot@TEST apache-storm-0.9.1-incubating]$ storm ui >/dev/null 2>&1 & [3] 3099 [irobot@TEST apache-storm-0.9.1-incubating]$ storm logviewer > /dev/null 2>&1 & [4] 3157 [irobot@TEST apache-storm-0.9.1-incubating]$ [irobot@TEST apache-storm-0.9.1-incubating]$ [irobot@TEST apache-storm-0.9.1-incubating]$ jps 6246 supervisor 6351 core 6065 nimbus 5810 QuorumPeerMain 6574 Jps 6472 logviewer >>> 查看日志位置：[irobot@TEST logs]$ pwd/home/irobot/storm/apache-storm-0.9.1-incubating/logs[irobot@TEST logs]$ ls -altotal 76drwxr-xr-x 2 sms oinstall 4096 May 26 08:22 .drwxrwxrwx 9 sms oinstall 4096 May 26 08:09 ..-rw-r--r-- 1 sms oinstall 0 May 26 08:07 access.log-rw-r--r-- 1 sms oinstall 219 May 26 08:22 logviewer.log-rw-r--r-- 1 sms oinstall 0 May 26 08:07 metrics.log-rw-r--r-- 1 sms oinstall 44020 May 26 08:18 nimbus.log-rw-r--r-- 1 sms oinstall 15590 May 26 08:20 supervisor.log-rw-r--r-- 1 sms oinstall 219 May 26 08:21 ui.log[irobot@TEST apache-storm-0.9.1-incubating]$ cd logs[irobot@TEST logs]$ lsaccess.log logviewer.log metrics.log nimbus.log supervisor.log ui.log[irobot@TEST logs]$ tail -f *.log==> nimbus.log <==2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:host.name=TEST2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.6.0_212015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Sun Microsystems Inc.2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.home=/u01/app/software/jdk1.6.0_21/jre2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.class.path=/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-codec-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/snakeyaml-1.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5-20081211.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-time-0.4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/zookeeper-3.3.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/core.incubator-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-core-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-io-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/math.numeric-tower-0.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/storm-core-0.9.1-incubating.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/reflectasm-1.07-shaded.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/meat-locker-0.3.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-exec-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-core-1.1.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-servlet-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/minlog-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/slf4j-api-1.6.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-fileupload-1.2.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/hiccup-0.3.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/guava-13.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/carbonite-1.3.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clojure-1.4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/netty-3.6.3.Final.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-lang-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-devel-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jline-2.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jgrapht-core-0.9.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/disruptor-2.10.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/kryo-2.17.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-util-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-client-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpcore-4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-framework-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/joda-time-2.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/objenesis-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.cli-0.2.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/asm-4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.logging-0.2.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-logging-1.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/json-simple-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/compojure-1.1.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clout-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-classic-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/junit-3.8.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.macro-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-stacktrace-0.2.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpclient-4.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd642015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:os.version=2.6.32-431.el6.x86_642015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:user.name=sms2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/irobot2015-05-27 09:17:32 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:host.name=TEST2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.version=1.6.0_212015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.vendor=Sun Microsystems Inc.2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.home=/u01/app/software/jdk1.6.0_21/jre2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.class.path=/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-codec-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/snakeyaml-1.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5-20081211.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-time-0.4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/zookeeper-3.3.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/core.incubator-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-core-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-io-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/math.numeric-tower-0.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/storm-core-0.9.1-incubating.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/reflectasm-1.07-shaded.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/meat-locker-0.3.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-exec-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-core-1.1.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-servlet-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/minlog-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/slf4j-api-1.6.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-fileupload-1.2.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/hiccup-0.3.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/guava-13.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/carbonite-1.3.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clojure-1.4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/netty-3.6.3.Final.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-lang-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-devel-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jline-2.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jgrapht-core-0.9.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/disruptor-2.10.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/kryo-2.17.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-util-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-client-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpcore-4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-framework-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/joda-time-2.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/objenesis-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.cli-0.2.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/asm-4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.logging-0.2.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-logging-1.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/json-simple-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/compojure-1.1.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clout-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-classic-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/junit-3.8.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.macro-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-stacktrace-0.2.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpclient-4.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.io.tmpdir=/tmp2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.compiler=<NA>2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.name=Linux2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.arch=amd642015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.version=2.6.32-431.el6.x86_642015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.name=sms2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.home=/home/irobot2015-05-27 09:17:32 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.dir=/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:17:34 b.s.d.nimbus [INFO] Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, "topology.skip.missing.kryo.registrations" false, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 500, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/home/irobot/storm/stormlocaldir", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "192.168.1.100", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["192.168.1.100"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 100, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8088, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed", "topology.optimize" true, "topology.max.task.parallelism" nil}2015-05-27 09:17:34 b.s.d.nimbus [INFO] Using default scheduler2015-05-27 09:17:34 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting2015-05-27 09:17:34 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.100:2181 sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@e5f83682015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] Opening socket connection to server /192.168.1.100:21812015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] Socket connection established to TEST/192.168.1.100:2181, initiating session2015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] Session establishment complete on server TEST/192.168.1.100:2181, sessionid = 0x14d92ef29b10000, negotiated timeout = 200002015-05-27 09:17:34 b.s.zookeeper [INFO] Zookeeper state update: :connected:none2015-05-27 09:17:34 o.a.z.ZooKeeper [INFO] Session: 0x14d92ef29b10000 closed2015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] EventThread shut down2015-05-27 09:17:34 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting2015-05-27 09:17:34 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.100:2181/storm sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@75a704762015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] Opening socket connection to server /192.168.1.100:21812015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] Socket connection established to TEST/192.168.1.100:2181, initiating session2015-05-27 09:17:34 o.a.z.ClientCnxn [INFO] Session establishment complete on server TEST/192.168.1.100:2181, sessionid = 0x14d92ef29b10001, negotiated timeout = 200002015-05-27 09:17:34 b.s.d.nimbus [INFO] Starting Nimbus server...==> supervisor.log <==2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:host.name=TEST2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.6.0_212015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Sun Microsystems Inc.2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.home=/u01/app/software/jdk1.6.0_21/jre2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.class.path=/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-codec-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/snakeyaml-1.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5-20081211.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-time-0.4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/zookeeper-3.3.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/core.incubator-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-core-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-io-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/math.numeric-tower-0.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/storm-core-0.9.1-incubating.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/reflectasm-1.07-shaded.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/meat-locker-0.3.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-exec-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-core-1.1.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-servlet-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/minlog-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/slf4j-api-1.6.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-fileupload-1.2.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/hiccup-0.3.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/guava-13.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/carbonite-1.3.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clojure-1.4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/netty-3.6.3.Final.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-lang-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-devel-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jline-2.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jgrapht-core-0.9.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/disruptor-2.10.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/kryo-2.17.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-util-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-client-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpcore-4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-framework-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/joda-time-2.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/objenesis-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.cli-0.2.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/asm-4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.logging-0.2.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-logging-1.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/json-simple-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/compojure-1.1.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clout-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-classic-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/junit-3.8.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.macro-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-stacktrace-0.2.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpclient-4.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd642015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:os.version=2.6.32-431.el6.x86_642015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:user.name=sms2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/irobot2015-05-27 09:19:34 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:host.name=TEST2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.version=1.6.0_212015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.vendor=Sun Microsystems Inc.2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.home=/u01/app/software/jdk1.6.0_21/jre2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.class.path=/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-codec-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/snakeyaml-1.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5-20081211.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-time-0.4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/zookeeper-3.3.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/core.incubator-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-core-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-io-1.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/math.numeric-tower-0.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/storm-core-0.9.1-incubating.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/reflectasm-1.07-shaded.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/meat-locker-0.3.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-exec-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-core-1.1.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-servlet-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/minlog-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/slf4j-api-1.6.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-fileupload-1.2.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/hiccup-0.3.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/guava-13.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/carbonite-1.3.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clojure-1.4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/netty-3.6.3.Final.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-lang-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-devel-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jline-2.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jgrapht-core-0.9.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/disruptor-2.10.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/kryo-2.17.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-util-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-client-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpcore-4.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/curator-framework-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/joda-time-2.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/objenesis-1.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.cli-0.2.2.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/asm-4.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.logging-0.2.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/commons-logging-1.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/json-simple-1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/compojure-1.1.3.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/servlet-api-2.5.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/jetty-6.1.26.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clout-1.0.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/logback-classic-1.0.6.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/junit-3.8.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/tools.macro-0.1.0.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/clj-stacktrace-0.2.4.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/lib/httpclient-4.1.1.jar:/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.io.tmpdir=/tmp2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.compiler=<NA>2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.name=Linux2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.arch=amd642015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.version=2.6.32-431.el6.x86_642015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.name=sms2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.home=/home/irobot2015-05-27 09:19:34 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.dir=/home/irobot/storm/apache-storm-0.9.1-incubating/conf2015-05-27 09:19:37 b.s.d.supervisor [INFO] Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, "topology.skip.missing.kryo.registrations" false, "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 500, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/home/irobot/storm/stormlocaldir", "storm.messaging.netty.buffer_size" 5242880, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "192.168.1.100", "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, "storm.zookeeper.servers" ["192.168.1.100"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 100, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8088, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed", "topology.optimize" true, "topology.max.task.parallelism" nil}2015-05-27 09:19:37 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting2015-05-27 09:19:37 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.100:2181 sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@1d1e082e2015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] Opening socket connection to server /192.168.1.100:21812015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] Socket connection established to TEST/192.168.1.100:2181, initiating session2015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] Session establishment complete on server TEST/192.168.1.100:2181, sessionid = 0x14d92ef29b10002, negotiated timeout = 200002015-05-27 09:19:37 b.s.zookeeper [INFO] Zookeeper state update: :connected:none2015-05-27 09:19:37 o.a.z.ZooKeeper [INFO] Session: 0x14d92ef29b10002 closed2015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] EventThread shut down2015-05-27 09:19:37 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting2015-05-27 09:19:37 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.100:2181/storm sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@4413d0522015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] Opening socket connection to server /192.168.1.100:21812015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] Socket connection established to TEST/192.168.1.100:2181, initiating session2015-05-27 09:19:37 o.a.z.ClientCnxn [INFO] Session establishment complete on server TEST/192.168.1.100:2181, sessionid = 0x14d92ef29b10003, negotiated timeout = 200002015-05-27 09:19:37 b.s.d.supervisor [INFO] Starting supervisor with id 14108489-26ad-4943-9b57-c80093ea9399 at host TEST==> ui.log <==2015-05-27 09:21:29 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog2015-05-27 09:21:29 o.m.log [INFO] jetty-6.1.262015-05-27 09:21:29 o.m.log [INFO] Started SocketConnector@0.0.0.0:80882015-05-27 09:21:49 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.100:6627==> logviewer.log <==2015-05-27 09:22:18 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog2015-05-27 09:22:18 o.m.log [INFO] jetty-6.1.262015-05-27 09:22:18 o.m.log [INFO] Started SocketConnector@0.0.0.0:80005. run examples [refer to https://github.com/apache/storm/tree/master/examples/storm-starter] [irobot@TEST storm]$ pwd/home/irobot/storm[irobot@TEST storm]$ git clone git://github.com/apache/storm.git[irobot@TEST storm]$ ls -altotal 32500drwxr-xr-x 7 sms oinstall 4096 May 26 09:09 .drwx------. 50 sms oinstall 4096 May 26 08:09 ..drwxrwxrwx 9 sms oinstall 4096 May 26 08:09 apache-storm-0.9.1-incubating-rw-r--r-- 1 sms oinstall 15544397 May 25 18:05 apache-storm-0.9.1-incubating.zipdrwxr-xr-x 14 sms oinstall 4096 May 26 09:13 stormdrwxr-xr-x 8 sms oinstall 4096 May 25 18:07 storm-apache-0.9.1drwxr-xr-x 3 sms oinstall 4096 May 25 15:43 zksnapshotdrwxr-xr-x 10 sms oinstall 4096 May 25 15:43 zookeeper-3.4.6-rw-r--r-- 1 sms oinstall 17699306 Oct 31 2014 zookeeper-3.4.6.tar.gz[irobot@TEST storm]$ cd storm[irobot@TEST storm]$ lsbin conf dev-tools doap_Storm.rdf examples KEYS logback pom.xml SECURITY.md storm-core storm-multilang TODOCHANGELOG.md DEVELOPER.md DISCLAIMER docs external LICENSE NOTICE README.markdown storm-buildtools storm-dist STORM-UI-REST-API.md VERSION[irobot@TEST storm]$ pwd/home/irobot/storm/storm (可选) 如果想试试重新编译所有的示例，可以这么做 [irobot@TEST storm]$ mvn clean install -DskipTests=true INFO] Scanning for projects... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] Storm [INFO] maven-shade-clojure-transformer [INFO] storm-maven-plugins [INFO] multilang-javascript [INFO] multilang-python [INFO] multilang-ruby [INFO] Storm Core [INFO] storm-starter [INFO] storm-kafka [INFO] storm-hdfs [INFO] storm-hbase [INFO] storm-hive [INFO] storm-jdbc [INFO] storm-redis [INFO] storm-eventhubs [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Storm 0.11.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ storm --- [INFO] Executed tasks [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ storm-eventhubs --- [INFO] Installing /home/irobot/storm/storm/external/storm-eventhubs/target/storm-eventhubs-0.11.0-SNAPSHOT.jar to /home/irobot/.m2/repository/org/apache/storm/storm-eventhubs/0.11.0-SNAPSHOT/storm-eventhubs-0.11.0-SNAPSHOT.jar [INFO] Installing /home/irobot/storm/storm/external/storm-eventhubs/pom.xml to /home/irobot/.m2/repository/org/apache/storm/storm-eventhubs/0.11.0-SNAPSHOT/storm-eventhubs-0.11.0-SNAPSHOT.pom [INFO] Installing /home/irobot/storm/storm/external/storm-eventhubs/target/storm-eventhubs-0.11.0-SNAPSHOT-jar-with-dependencies.jar to /home/irobot/.m2/repository/org/apache/storm/storm-eventhubs/0.11.0-SNAPSHOT/storm-eventhubs-0.11.0-SNAPSHOT-jar-with-dependencies.jar [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Storm ............................................. SUCCESS [1.681s] [INFO] maven-shade-clojure-transformer ................... SUCCESS [2.153s] [INFO] storm-maven-plugins ............................... SUCCESS [1.866s] [INFO] multilang-javascript .............................. SUCCESS [0.204s] [INFO] multilang-python .................................. SUCCESS [0.162s] [INFO] multilang-ruby .................................... SUCCESS [0.162s] [INFO] Storm Core ........................................ SUCCESS [2:24.504s] [INFO] storm-starter ..................................... SUCCESS [7.510s] [INFO] storm-kafka ....................................... SUCCESS [0.899s] [INFO] storm-hdfs ........................................ SUCCESS [1.448s] [INFO] storm-hbase ....................................... SUCCESS [1.859s] [INFO] storm-hive ........................................ SUCCESS [38.793s] [INFO] storm-jdbc ........................................ SUCCESS [2:23.753s] [INFO] storm-redis ....................................... SUCCESS [1:13.365s] [INFO] storm-eventhubs ................................... SUCCESS [1:17.599s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 8:16.841s [INFO] Finished at: Tue May 26 17:25:11 CST 2015 [INFO] Final Memory: 114M/734M [INFO] ------------------------------------------------------------------------[irobot@TEST storm]$[irobot@TEST storm]$[irobot@TEST storm]$ cd examples/storm-starter[irobot@TEST storm-starter]$ lsmultilang pom.xml README.markdown src target test[irobot@TEST storm-starter]$使用maven编译storm_starter源码 (备注：storm_starter源码的其他下载地址：git clone https://github.com/nathanmarz/storm-starter.git )编译：mvn compile打包：mvn package删除：mvn clean // mvn clean install -Dstorm.topology=TopologyMain// storm jar storm-word-count-1.0-SNAPSHOT-jar-with-dependencies.jar TopologyMain 1.txt WordCount运行例子： All following examples require that you run cd examples/storm-starter beforehand. [irobot@TEST storm]$ cd examples/storm-starter [irobot@TEST storm-starter]$ mvn clean install -DskipTests=true 或者 只编译： compile WordCountTopology , use the command: [irobot@TEST storm-starter]$ mvn clean compile install -Dstorm.topology=storm.starter.WordCountTopology 运行: Run the WordCountTopology in local mode (LocalCluster) [irobot@TEST target]$ pwd /home/irobot/storm/storm/examples/storm-starter/target [irobot@TEST target]$ storm jar storm-starter-*.jar storm.starter.WordCountTopology WordCount ---------------------------------------------------------------- 可能会碰到执行问题：原因是 apache-storm-0.9.1-incubating 可能不支持 某些例子中的方法。 [irobot@TEST starter]$ pwd /home/irobot/storm/storm/examples/storm-starter/src/jvm/storm/starter [irobot@TEST starter]$ ls BasicDRPCTopology.java ManualDRPC.java RollingTopWords.java spout TransactionalWords.java WordCountTopology.java bolt PrintSampleStream.java SingleJoinExample.java tools trident WordCountTopologyNode.java ExclamationTopology.java ReachTopology.java SkewedRollingTopWords.java TransactionalGlobalCount.java util [irobot@TEST starter]$ cat WordCountTopology.java //注释掉以下代码，替换成submitTopology方法。 然后重新编译 //StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology()); StormSubmitter.submitTopology(args[0], conf, builder.createTopology()); ----------------------------------------------------------------查看： Storm UI： http://192.168.1.100:8088/ 0 0
- Storm安装介绍
- storm介绍
- storm介绍
- Storm介绍
- Storm介绍
- storm介绍
- Storm介绍
- storm 介绍
- storm介绍
- 【storm】storm集群安装
- Storm学习(一)Storm介绍
- Storm学习(一) Storm介绍
- Storm学习(一)Storm介绍
- Storm学习(一)Storm介绍
- Storm安装
- Storm安装
- storm安装
- storm-安装
- 交换机端口untaged、taged、trunk、access 的区别
- hdu 1325 Is It A Tree? 并查集
- rpc编程常见问题
- 详解JMeter正则表达式
- 写给初学者的6条网页设计安全配色指南
- Storm安装介绍
- java_集合框架基础总结
- HDU 1257 最少拦截系统
- 缓存技术PK：选择Memcached还是Redis？
- 【转】计算机视觉基础2——相机成像的几何描述
- 【HTML5+CSS3开发实战】第2章 初识HTML5
- 自定义类似RadioButton的单选、多选的ListView
- cpu亲和力总结taskset和setcpu及其他相关
- 为什么两个float类型的数值不能直接相等
