centos的twitter storm安装和storm-start的本地运行

这里介绍一下storm的基本安装和不用lein的storm-starter运行方法。

一. Storm及相关软件安装

1. 安装python2.7.2

============================

# wget http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz

# tar zxvf Python-2.7.2.tgz

# cd Python-2.7.2

# ./configure

# make

# make install

# vi /ect/ld.so.conf

--

文件末尾追加

/usr/local/lib

--

#sudo ldconfig

============================

2. 安装zookeeper

============================

# wget http://ftp.meisei-u.ac.jp/mirror/apache/dist//zookeeper/zookeeper-3.3.3/zookeeper-3.3.3.tar.gz

# tar zxf zookeeper-3.3.3.tar.gz

# cp -R zookeeper-3.3.3 /usr/local/

# ln -s /usr/local/zookeeper-3.3.3/ /usr/local/zookeeper

设置 ZOOKEEPER_HOME 和$ZOOKEEPER_HOME/bin

#vi ~/.bashrc

--

追加

export ZOOKEEPER_HOME="/path/to/zookeeper"

export PATH=$PATH:$ZOOKEEPER_HOME/bin

--

用zoo_sample.cfg制作$ZOOKEEPER_HOME/conf/zoo.cfg

#cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg

制作dataLogDir和运行Dir

sudo mkdir /tmp/zookeeper

sudo mkdir /var/log/zookeeper

============================

3. 安装zeromq & jzmq

============================

1）安装zeromq

# wget http://download.zeromq.org/historic/zeromq-2.1.7.tar.gz

# tar zxf zeromq-2.1.7.tar.gz

# cd zeromq-2.1.7

# ./configure

＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊

如果报错

configure: error: in `/usr/local/download/zeromq-2.1.7':

configure: error: no acceptable C compiler found in $PATH

See `config.log' for more details

原因为没有安装c compiler

解决方法

# yum install gcc*

如果遇到Error:cannot link with -luuid, install uuid-dev

原因为缺少uuid相关package

解决方法

# yum install uuid*

# yum install e2fsprogs*

# yum install libuuid*

＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊

#make

#make install

更新LD_LIBRARY_PATH

#ldconfig

============================

2）安装jzmq

============================

#yum install git

#git clone git://github.com/nathanmarz/jzmq.git

＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊

如果有proxy的话执行

#git clone http://github.com/nathanmarz/jzmq.git

就能避开git的代理设置

＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊

# cd jzmq

# ./autogen.sh

＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊

如果报错

autogen.sh:error: could not find libtool. libtool is required to run autogen.sh.

缺少libtool

# yum install libtool*

＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊

# ./configure

# make

# make install

============================

4. 安装storm

============================

# wget https://github.com/downloads/nathanmarz/storm/storm-0.7.1.zip

# unzip storm- 0.7.1 zip

# cp -R storm-0.7.1 /usr/local/

# vi ～/.bashrc

--

追加

export STORM_HOME=/usr/local/storm-0.7.1

export PATH=$PATH:$STORM_HOME/bin

--

============================

到此为止，storm安装完毕。

二. 本地运行测试程序storm-start

按照https://github.com/nathanmarz/storm-starter，执行这个程序需要用lein，这里介绍的方法用eclipse代替lein的作用。

1)  安装twitter4j

# mkdir twitter4j

# cd twitter4j

# wget http://twitter4j.org/en/twitter4j-2.2.5.zip

# unzip twitter4j-2.2.5.zip

2） 追加源文件storm-start/src/jvm/storm

  使用eclipse建立java project。追加twitter4j和storm的jar文件。

  File-> New -> Java Project ->随便取个名字-> Next -> Libraries -> add External JARs...-> 追加twitter4j和storm的jar文件（/path/to/twitter4j/lib/*.jar和/path/to/storm/lib/*.jar和/path/to/storm/storm-{version}.jar）-> Finsh

  导入storm-start

  File -> Import -> General -> File System -> Next -> Browse(From directory) -> /path/to/storm-start/src/jvm/storm -> Browse(Info floder) -> xxx -> src -> OK -> “storm” 和 “Create top-level folder”前打勾 -> Finish

完成之后如图：

3） 追加源文件storm-start/multilang/resources×（python 文件word count用）

  File -> Import -> General -> File System -> Next -> Browse(From directory) -> /path/to/storm-start/multilang/resources -> Browse(Info floder) -> xxx -> OK -> check “resources” and “Create top-level folder” -> Finish

2个源文件都追加好之后，eclipse左边显示如下图：

4） JAR export

  File -> Export -> JAR -> JAR file -> 取消 “.classpath” ，“.project” 和 “<.settings” ->的勾 browse -> path/to/export/name.jar -> Finish (忽视 warnings)

4. 执行刚才编译的文件。

# storm jar StormStarter.jar storm.starter.ExclamationTopology

如果出现类似下面的文字，说明运行成功！

....

11367 [Thread-25] INFO  backtype.storm.daemon.task  - Emitting: class storm.starter.ExclamationTopology$ExclamationBolt source: 2:3, stream: 1, id: {}, [golda!!!]

....

参考链接：https://github.com/nathanmarz

            http://tjun.jp/blog/category/storm/