大数据平台搭建之components版本选择

确定依据：CDH5内的所有版本都兼容

JDK:

same one with CDH 5.4.X release: Oracle JDK 1.7.0_75 

Open Source components versions:

Based on: CDH 5 (storm to be determined), cdh5-*_5.4.1(latest github branch)

Components name：spark, hadoop, hive, kafka, sqoop, flume, zookeeper, storm, hbase

Components Project(source code)gittar ball/ jar ball(jenerated by jenkins)mvn repo(both 2)Build jobsjenkins

Project location: git

tar ball/ jar ball localtion: mvn nexus repository

Build jobs(generate jar and image): jenkins

Following components has cdh5-*_5.4.1 branch:

1. spark 1.3.0: git clone https://github.com/cloudera/spark.git
   spark sql
2. hive 1.1.0: https://github.com/cloudera/hive.git
3. sqoop 1.4.5: https://github.com/cloudera/sqoop.git
4. flume 1.5.0: https://github.com/cloudera/flume-ng.git
5. zookeeper 3.4.5: https://github.com/cloudera/zookeeper.git
6. hbase 1.0.0: https://github.com/cloudera/hbase.git
7. hadoop 2.6.0 from cdh 5.4.1: http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.4.1-src.tar.gz

Others:

• kafka(cdh5-0.8.2.0_1.3.0, also from cdh5, should be ok): https://github.com/cloudera/kafka.git
• storm(cdh not support, the version will based on how we use storm): https://github.com/apache/storm.git
  Latest storm-0.9.4 works with zookeeper-3.4.6, hbase-0.98.1, hadoop-2.2.0, kafka-0.8.1.1