在Ubuntu中安装HBase

来与：http://blog.fens.me/linux-hbase-install/

前言

HBase是Hadoop家族中的一个分布式数据库产品，HBase支持高并发读写，列式数据存储，高效的索引，自动分片，自动Region迁移等许多优点，已经越来越多的被界业认可并实施。

目录

1. 在Ubuntu中环境准备
2. HBase安装
3. Thrift安装

1 在Ubuntu中环境准备

HBase是基于Java开发的运行Hadoop平台上分布式NoSQL数据库软件，HBase没有提供Windows系统安装版本。我在这里也只介绍HBase在Linux Ubuntu系统中的安装。

由于HBase是运行在Hadoop平台上面的，因此我们需要先安装好Hadoop的环境，Hadoop的安装请参考文章：[Hadoop历史版本安装](http://blog.fens.me/hadoop-history-source-install/)

HBase没有提供apt的软件源安装，我们需要自己去官方网络下载HBase软件包进行安装。HBase下载页：http://www.apache.org/dyn/closer.cgi/hbase/

系统环境：

• Linux Ubuntu 12.04.2 LTS 64bit server
• Java JDK 1.6.0_45
• Hadoop 1.1.2

2 HBase安装

2.1 下载HBase

# 通过wget命令下载~ wget http://www.gaidso.com/apache/hbase/stable/hbase-0.94.18.tar.gz# 解压HBase~ tar xvf hbase-0.94.18.tar.gz# 移动HBase目录到文件夹~ mv hbase-0.94.18/ /home/conan/hadoop/# 进入目录~ cd /home/conan/hadoop/hbase-0.94.18

2.2 配置HBase

2.2.1 修改启动文件hbase-env.sh

~ vi conf/hbase-env.sh#打开注释export JAVA_HOME=/home/conan/toolkit/jdk16export HBASE_CLASSPATH=/home/conan/hadoop/hadoop-1.1.2/confexport HBASE_MANAGES_ZK=true

2.2.2 修改配置文件 hbase-site.xml

~ vi conf/hbase-site.xml<configuration><property><name>hbase.rootdir</name><value>hdfs://master:9000/hbase</value></property><property><name>hbase.cluster.distributed</name><value>true</value></property><property><name>dfs.replication</name><value>1</value></property><property><name>hbase.zookeeper.quorum</name><value>master</value></property><property><name>hbase.zookeeper.property.clientPort</name><value>2181</value></property><property><name>hbase.zookeeper.property.dataDir</name><value>/home/conan/hadoop/hdata</value></property></configuration>

复制hadoop环境的配置文件和类库

~ cp ~/hadoop/hadoop-1.1.2/conf/hdfs-site.xml conf/~ cp ~/hadoop/hadoop-1.1.2/hadoop-core-1.1.2.jar lib/~ mkdir /home/conan/hadoop/hdata

2.3 启动hadoop和hbase

~ /home/conan/hadoop/hadoop-1.1.2/bin/start-all.sh~ /home/conan/hadoop/hbase-0.94.18/bin/start-hbase.sh# 查看hbase进程~ jps13838 TaskTracker13541 JobTracker15946 HMaster16756 Jps12851 NameNode13450 SecondaryNameNode13133 DataNode15817 HQuorumPeer16283 HRegionServer

2.4 打开HBase命令行客户端访问Hbase

~ bin/hbase shellHBase Shell; enter 'help' for list of supported commands.Type "exit" to leave the HBase ShellVersion 0.94.18, r1577788, Sat Mar 15 04:46:47 UTC 2014hbase(main):002:0> helpHBase Shell, version 0.94.18, r1577788, Sat Mar 15 04:46:47 UTC 2014Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command.Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.COMMAND GROUPS:  Group name: general  Commands: status, version, whoami  Group name: ddl  Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, is_disabled, is_enabled, list, show_filters  Group name: dml  Commands: count, delete, deleteall, get, get_counter, incr, put, scan, truncate  Group name: tools  Commands: assign, balance_switch, balancer, close_region, compact, flush, hlog_roll, major_compact, move, split, unassign, zk_dump  Group name: replication  Commands: add_peer, disable_peer, enable_peer, list_peers, list_replicated_tables, remove_peer, start_replication, stop_replication  Group name: snapshot  Commands: clone_snapshot, delete_snapshot, list_snapshots, restore_snapshot, snapshot  Group name: security  Commands: grant, revoke, user_permissionSHELL USAGE:Quote all names in HBase Shell such as table and column names.  Commas delimitcommand parameters.  Type  after entering a command to run it.Dictionaries of configuration used in the creation and alteration of tables areRuby Hashes. They look like this:  {'key1' => 'value1', 'key2' => 'value2', ...}and are opened and closed with curley-braces.  Key/values are delimited by the'=>' character combination.  Usually keys are predefined constants such asNAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type'Object.constants' to see a (messy) list of all constants in the environment.If you are using binary keys or values and need to enter them in the shell, usedouble-quote'd hexadecimal representation. For example:  hbase> get 't1', "key\x03\x3f\xcd"  hbase> get 't1', "key\003\023\011"  hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added.For more on the HBase Shell, see http://hbase.apache.org/docs/current/book.html

2.5 HBase简单命令操作

#创建一个新表studenthbase(main):003:0> create 'student','info'0 row(s) in 1.2680 seconds#查看所有的表hbase(main):004:0> listTABLEstudent1 row(s) in 0.0330 seconds#查看student的表结构hbase(main):005:0> describe 'student'DESCRIPTION                                                 ENABLED 'student', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', true  BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS  => '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL = > '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE = > '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',  BLOCKCACHE => 'true'}1 row(s) in 0.1100 seconds#同student表中插入一条数据hbase(main):007:0> put 'student','mary','info:age','19'0 row(s) in 0.0490 seconds#从student表中取出mary的数据hbase(main):008:0> get 'student','mary'COLUMN                   CELL info:age                timestamp=1396366643298, value=191 row(s) in 0.0190 seconds#让student表失效hbase(main):009:0> disable 'student'0 row(s) in 1.2400 seconds#列出所有表hbase(main):010:0> listTABLEstudent1 row(s) in 0.0310 seconds#删除student表hbase(main):013:0>  drop 'student'0 row(s) in 1.1100 seconds#列出所有表hbase(main):014:0> listTABLE0 row(s) in 0.0400 seconds

3 Thrift安装

安装完成HBase后，我们还需要安装Thrift，因为其他语言调用HBase的时候，是通过Thrift连接的。

Thrift是需要本地编译的，官方没有提供二进制安装包，首先下载thrift-0.9.1，Thrift下载页：http://thrift.apache.org/download

3.1 下载thrift

下载Thrift有两种方式，直接下载源代码发行包，或者通过git下载源代码，请选择其中一种方式下载。

3.1.1 直接下载源代码发行包 thrift-0.9.1.tar.gz

~ wget http://apache.fayea.com/apache-mirror/thrift/0.9.1/thrift-0.9.1.tar.gz~ tar xvf thrift-0.9.1.tar.gz~ mv thrift-0.9.1/ /home/conan/hadoop/~ cd /home/conan/hadoop/

注：后文中的各种错误，都是这个包引起的

3.1.2 通过git下载源代码

~ git clone https://git-wip-us.apache.org/repos/asf/thrift.git thrift-git~ mv thrift-git/ /home/conan/hadoop/~ cd /home/conan/hadoop/

为了避免各种出错，建议使用git下载源代码安装

3.2 通过thrift-0.9.1.tar.gz 发行包安装Thrift

Thrift是需要本地编译的，在Thrift解压目录输入./configure，会列Thrift在当前机器所支持的语言环境。

3.2.1 安装Thrift的依赖包

sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev

如果只是为了连接rhbase，默认配置就可以了。如果除了希望支持rhbase访问，还支持PHP,Python,C++等语言的访问，就需要在系统中，装一些额外的类库。大家可以根据自己的要求，安装对应的软件包并设置Thrift的编译参数。

生成配置脚本

~  ./configure//省略部分日志输出thrift 0.9.1Building C++ Library ......... : yesBuilding C (GLib) Library .... : noBuilding Java Library ........ : yesBuilding C# Library .......... : noBuilding Python Library ...... : yesBuilding Ruby Library ........ : noBuilding Haskell Library ..... : noBuilding Perl Library ........ : noBuilding PHP Library ......... : noBuilding Erlang Library ...... : noBuilding Go Library .......... : noBuilding D Library ........... : noC++ Library:   Build TZlibTransport ...... : yes   Build TNonblockingServer .. : yes   Build TQTcpServer (Qt) .... : noJava Library:   Using javac ............... : javac   Using java ................ : java   Using ant ................. : /home/conan/toolkit/ant184/bin/antPython Library:   Using Python .............. : /usr/bin/pythonIf something is missing that you think should be present,please skim the output of configure to find the missingcomponent.  Details are present in config.log.

我本机的已支持C++, Java与Thrift的通信。

3.2.2 增加Python语言的通信

虽然Python已被显示支持与Thrift但在后面编译过程中，还是缺少一些Python的库，我们需要再增加Python的依赖库

安装Python的依赖包

sudo apt-get install python-all python-all-dev python-all-dbg

3.2.3 增加PHP语言的通信

安装PHP的依赖包

sudo apt-get install php5-dev php5-cli phpunit

生成配置脚本

~  ./configure --enable-thrift_protocol//省略部分日志输出thrift 0.9.1Building C++ Library ......... : yesBuilding C (GLib) Library .... : noBuilding Java Library ........ : yesBuilding C# Library .......... : noBuilding Python Library ...... : yesBuilding Ruby Library ........ : noBuilding Haskell Library ..... : noBuilding Perl Library ........ : noBuilding PHP Library ......... : yesBuilding Erlang Library ...... : noBuilding Go Library .......... : noBuilding D Library ........... : noC++ Library:   Build TZlibTransport ...... : yes   Build TNonblockingServer .. : yes   Build TQTcpServer (Qt) .... : noJava Library:   Using javac ............... : javac   Using java ................ : java   Using ant ................. : /home/conan/toolkit/ant184/bin/antPython Library:   Using Python .............. : /usr/bin/pythonPHP Library:   Using php-config .......... : /usr/bin/php-configIf something is missing that you think should be present,please skim the output of configure to find the missingcomponent.  Details are present in config.log.

我们看到Thrift的配置中，增加了对PHP语言的支持。

3.2.4 编译和安装

# 编译Thrift~ make//省略部分日志make[5]: 正在进入目录 `/home/conan/hadoop/thrift-0.9.1/lib/php/src/ext/thrift_protocol'make[5]: *** 没有指明目标并且找不到 makefile。 停止。make[5]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1/lib/php/src/ext/thrift_protocol'make[4]: *** [src/ext/thrift_protocol/modules/thrift_protocol.so] 错误 2make[4]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1/lib/php'make[3]: *** [all-recursive] 错误 1make[3]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1/lib/php'make[2]: *** [all-recursive] 错误 1make[2]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1/lib'make[1]: *** [all-recursive] 错误 1make[1]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1'make: *** [all] 错误 2

在make生成过程，出现PHP的编译错误。从Thrift的错误列表中，我们可以找到错误描述( https://issues.apache.org/jira/browse/THRIFT-2265 )，这是由于Thrift-0.9.1发行包，打包时缺少了PHP扩展文件造成的错误，并在Thrift-0.9.2版本中修复。

所以，我们如果还想继续使用Thrift-0.9.1版本，则不能支持PHP语言。

# 生成配置信息，不包括PHP模块~ ./configure --without-php_extension# 编译Thrift~ make

编译过程中，又出现了C++编译错误。

Makefile:832: 警告：覆盖关于目标“gen-cpp/ThriftTest.cpp”的命令Makefile:829: 警告：忽略关于目标“gen-cpp/ThriftTest.cpp”的旧命令/bin/bash ../../libtool --tag=CXX   --mode=link g++ -Wall -g -O2 -L/usr/lib   -o libtestgencpp.la  ThriftTest_constants.lo ThriftTest_types.lo ../../lib/cpp/libthrift.la -lssl -lcrypto -lrt -lpthreadlibtool: link: ar cru .libs/libtestgencpp.a .libs/ThriftTest_constants.o .libs/ThriftTest_types.oar: .libs/ThriftTest_constants.o: No such file or directorymake[3]: *** [libtestgencpp.la] 错误 1make[3]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1/test/cpp'make[2]: *** [all-recursive] 错误 1make[2]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1/test'make[1]: *** [all-recursive] 错误 1make[1]:正在离开目录 `/home/conan/hadoop/thrift-0.9.1'make: *** [all] 错误 2

对于上面的2个编译错误，我决定换成git源代码的版本重新操作。

3.2 通过git源代码安装Thrift

运行安装命令

# 进行thrift-git目录~ cd /home/conan/hadoop/thrift-git# 复制0.9.1标签到新分支thrift-0.9.1~ git checkout -b thrift-0.9.1 0.9.1# 产生配置脚本~ ./bootstrap.sh# 生成配置信息~ ./configure# 编译Thrift~ make# 安装Thrift~ sudo make install

走了许多弯路，终于使用git源代码版本安装好了Thrift。

查看thrift版本

~ thrift -versionThrift version 0.9.1

接下来，我们启动HBase的Thrift Server服务

# 启动HBase的Thrift服务~ /home/conan/hadoop/hbase-0.94.18/bin/hbase-daemon.sh start thriftstarting thrift, logging to /home/conan/hadoop/hbase-0.94.18/bin/../logs/hbase-conan-thrift-master.out# 检查系统进程~ jps13838 TaskTracker13541 JobTracker15946 HMaster32120 Jps12851 NameNode13450 SecondaryNameNode13133 DataNode32001 ThriftServer15817 HQuorumPeer16283 HRegionServer

我们看到ThriftServer已被启动，后面我们就可以使用多种语言，通过Thrift来访问HBase了，这样就完成了HBase的安装。

转载请注明出处：
http://blog.fens.me/linux-hbase-install/

0