HBase0.94和Hadoop2 结合
来源:互联网 发布:linux组织文件目录结构 编辑:程序博客网 时间:2024/03/29 01:15
原文地址:https://hbase.apache.org/book.html
4.1.1. Apache HBase 0.94 with Hadoop 2
To get 0.94.x to run on Hadoop 2.2.0, you need to change the hadoop 2 and protobuf versions in the pom.xml: Here is a diff with pom.xml changes:
$ svn diff pom.xmlIndex: pom.xml===================================================================--- pom.xml (revision 1545157)+++ pom.xml (working copy)@@ -1034,7 +1034,7 @@ <slf4j.version>1.4.3</slf4j.version> <log4j.version>1.2.16</log4j.version> <mockito-all.version>1.8.5</mockito-all.version>- <protobuf.version>2.4.0a</protobuf.version>+ <protobuf.version>2.5.0</protobuf.version> <stax-api.version>1.0.1</stax-api.version> <thrift.version>0.8.0</thrift.version> <zookeeper.version>3.4.5</zookeeper.version>@@ -2241,7 +2241,7 @@ </property> </activation> <properties>- <hadoop.version>2.0.0-alpha</hadoop.version>+ <hadoop.version>2.2.0</hadoop.version> <slf4j.version>1.6.1</slf4j.version> </properties> <dependencies>
4.1.2. Apache HBase 0.92 and 0.94
HBase 0.92 and 0.94 versions can work with Hadoop versions, 0.20.205, 0.22.x, 1.0.x, and 1.1.x. HBase-0.94 can additionally work with Hadoop-0.23.x and 2.x, but you may have to recompile the code using the specific maven profile (see top level pom.xml)
4.1.3. Apache HBase 0.96
As of Apache HBase 0.96.x, Apache Hadoop 1.0.x at least is required. Hadoop 2 is strongly encouraged (faster but also has fixes that help MTTR). We will no longer run properly on older Hadoops such as 0.20.205 or branch-0.20-append. Do not move to Apache HBase 0.96.x if you cannot upgrade your Hadoop. See HBase, mail # dev - DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?
4.1.4. Hadoop versions 0.20.x - 1.x
DO NOT use Hadoop versions older than 2.2.0 for HBase versions greater than 1.0. Check release documentation if you are using an older version of HBase for Hadoop related information.
4.1.5. Apache HBase on Secure Hadoop
Apache HBase will run on any Hadoop 0.20.x that incorporates(合并) Hadoop security features as long as you do as suggested above and replace the Hadoop jar that ships with HBase with the secure version. If you want to read more about how to setup Secure HBase, see hbase.secure.configuration.
4.1.6. dfs.datanode.max.transfer.threads
An HDFS DataNode has an upper bound on the number of files that it will serve at any one time. Before doing any loading, make sure you have configured Hadoop’sconf/hdfs-site.xml, setting thedfs.datanode.max.transfer.threads value to at least the following:
<property> <name>dfs.datanode.max.transfer.threads</name> <value>4096</value></property>
Be sure to restart your HDFS after making the above configuration.
Not having this configuration in place makes for strange-looking failures. One manifestation is a complaint about missing blocks. For example:
10/12/08 20:10:31 INFO hdfs.DFSClient: Could not obtain block
blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException: No live nodes
contain current block. Will get new block locations from namenode and retry...
See also casestudies.max.transfer.threads and note that this property was previously known asdfs.datanode.max.xcievers (e.g.Hadoop HDFS: Deceived by Xciever).
- HBase0.94和Hadoop2 结合
- Centos 7 伪分布式安装Hadoop2.6和Hbase0.94
- hadoop2.20 和 hbase0.96 metrics 配置
- hbase0.94.14+hadoop2.6.0
- hadoop2.2.0 和 hbase0.94.12 平台使用phoenix
- hadoop2.5.0和hbase0.98完全分布式安装
- Hadoop2.7.1和Hbase0.98添加LZO压缩
- Hbase0.96.0 +hadoop2.2.0安装
- hadoop2.4.1+hbase0.98集群
- hadoop2.5.1+Hbase0.98安装
- hadoop2.2.0 hbase0.96 安装snappy
- hadoop2.2+hbase0.96+zookeeper3.4.5
- Hadoop2.2.0 + HBase0.96.1.1部署实践
- 从hadoop2.2,HBase0.96 mapreduce操作
- HBase0.96.0 for Hadoop2.2.0编译
- Phoenix实战(hadoop2、hbase0.96)
- hadoop2.4及hbase0.98安装入门
- HBase0.96.0 for Hadoop2.2.0编译
- [Usaco08Oct] Watering Hole
- 生命的力量
- 【NOIP 模拟题】[T2] 王者荣耀(二分答案+dp)
- Mysql 5.7.9以上版本 设置账户密码及授权
- Linux-CentOS6.7——安装Samba服务器
- HBase0.94和Hadoop2 结合
- LeetCode----sum-root-to-leaf-numbers
- 深入理解 MessageQueue , Message , Looper , Handler
- java使用url读取网页资源
- 仿美团首页功能选择的实现
- iostream流库
- hdu 2642 stars
- AIDL详解
- Retrofit的初步使用