storm cluster
来源:互联网 发布:漏洞网站源码 编辑:程序博客网 时间:2024/05/21 00:17
一、centos6.4 install virtualbox1.download rpm:$>:wget http://download.virtualbox.org/virtualbox/4.2.16/VirtualBox-4.2-4.2.16_86992_el6-1.x86_64.rpm2.install SDL:$>:yum install *SDL*3.install vbox:$>:rpm -ivh VirtualBox-4.2-4.2.16_86992_el6-1.x86_64.rpm二、install centos to vbox(NOT SUPPORT VITUAL TECHNOLOGY) 1.iso:download centos6.4_x86 2.install to vobx三、soft update and install 1.$>:yum update -y 2.$>:install jdk7 $>:unpack200 tools.pack tools.jar $>:unpack200 rt.pack rt.jar 3.set env四、vmware,4 machine 1.startup when boot: $>:VBoxManage startvm centos1 --type headless $>:VBoxManage startvm centos2 --type headless 五、ssh without pwd 1.$>:touch ~/.ssh/authorized_keys $>:chmod 600 ~/.ssh/authorized_keys $>:cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys六、install ZeroMQ and jzmq see:https://github.com/nathanmarz/storm/wiki/Installing-native-dependencies七、ditribute storm release v10.8.2 to all machines $>:scp storm-0.8.2.zip qibaoyuan@10.63.0.215:/home/qibaoyuan/soft 八、configure zookeeper and storm.yaml see:http://blog.linezing.com/2013/01/how-to-install-and-deploy-storm-cluster set up three :m183,m171,m172 $>:cp conf/defaults.yaml ~/.storm/storm.yaml ========~/.storm/storm.yaml===== storm.zookeeper.servers: - "m183" - "m171" - "m172" nimbus.host: "m183" =================================九、startup nimbus and supervisor on the machines,see ui:http://10.63.0.183:8080/ auto: ==crontab==== @reboot sh /home/qibaoyuan/soft/zookeeper-3.4.5/bin/zkServer.sh start @reboot source /etc/profile && cd ~ && rm -rf storm-local && storm supervisor > /dev/null 2>&1 & ==========十、submit to cluster 1.mvn storm-starter 2.scp to m183 3.in m183: $>:storm jar storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology WordCount十一、drpc 1.server only on m183:storm drpc 2.on other machine:set drpc.servers: -"m183" 3.submit job on drpc server: storm jar storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.ReachTopology reachTopology2