python 访问hbase
来源:互联网 发布:switch case用法 java 编辑:程序博客网 时间:2024/05/01 09:27
通过thrift,我们可以使用python访问hbase。
关于thrift
thrift是一个跨语言服务的软件开发框架(Thrift is a software framework for scalable cross-language services development.)。
它的官方网站是:http://incubator.apache.org/thrift/
1 下载thrift
svn co http://svn.apache.org/repos/asf/incubator/thrift/trunk thrift
基本安装工具环境yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel./configure 配置 ,看到如下信息:
thrift 0.9.1
Building C++ Library ......... : no
Building C (GLib) Library .... : no
Building Java Library ........ : yes
Building C# Library .......... : no
Building Python Library ...... : yes
Building Ruby Library ........ : no
Building Haskell Library ..... : no
Building Perl Library ........ : no
Building PHP Library ......... : no
Building Erlang Library ...... : no
Building Go Library .......... : no
Building D Library ........... : no
thrift 0.9.1
Building C++ Library ......... : no
Building C (GLib) Library .... : no
Building Java Library ........ : yes
Building C# Library .......... : no
Building Python Library ...... : yes
Building Ruby Library ........ : no
Building Haskell Library ..... : no
Building Perl Library ........ : no
Building PHP Library ......... : no
Building Erlang Library ...... : no
Building Go Library .......... : no
Building D Library ........... : no
Java Library:
Using javac ............... : javac
Using java ................ : java
Using ant ................. : /usr/bin/ant
Python Library:
Using Python .............. : /usr/bin/python
make
make install
查看thrift 版本
thrift -version
2. 安装python 的thrift 包
[root@YZSJHL19-81 py]# cd /opt/thrift-0.9.1/lib/py/
[root@YZSJHL19-81 py]# ls
build compat dist Makefile Makefile.am Makefile.in README setup.cfg setup.py src thrift.egg-info
[root@YZSJHL19-81 py]# python setup.py build
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.6
....................
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c src/protocol/fastbinary.c -o build/temp.linux-x86_64-2.6/src/protocol/fastbinary.o
gcc -pthread -shared build/temp.linux-x86_64-2.6/src/protocol/fastbinary.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/thrift/protocol/fastbinary.so
[root@YZSJHL19-81 py]# python setup.py install
running install
running bdist_egg
running egg_info
......
Adding thrift 0.9.1 to easy-install.pth file
Installed /usr/lib64/python2.6/site-packages/thrift-0.9.1-py2.6-linux-x86_64.egg
Processing dependencies for thrift==0.9.1
Finished processing dependencies for thrift==0.9.1
3 生成python和hbase通讯的thrift包:
[root@YZSJHL19-81 thrift]# cd /opt/hbase/src/main/resources/org/apache/hadoop/hbase/thrift
[root@YZSJHL19-81 thrift]# ls
Hbase.thrift
[root@YZSJHL19-81 thrift]# thrift --gen py:new_style Hbase.thrift
[root@YZSJHL19-81 thrift]# ls
gen-py Hbase.thrift
[root@YZSJHL19-81 thrift]# ll -t
总用量 28
drwxr-xr-x 3 root root 4096 1月 14 21:11 gen-py.bak
-rwxr-xr-x 1 hbase hbase 23434 8月 28 2012 Hbase.thrift
[root@YZSJHL19-81 thrift]# mv gen-py /usr/lib/python2.6/site-packages/
4 启动hbase 的thrift服务
hbase thrift -p 19090 -nonblocking start
5.编写测试程序
1 #!/usr/bin/python 2 #coding:utf8 3 #author:guoqiang.ma 4 #date:2014.1.15 5 import sys 6 #Hbase.thrift生成的py文件放在这里 7 sys.path.append('/usr/lib/python2.6/site-packages/gen-py') 8 from thrift import Thrift 9 from thrift.transport import TSocket 10 from thrift.transport import TTransport 11 from thrift.protocol import TBinaryProtocol 12 13 from hbase import Hbase 14 #如ColumnDescriptor 等在hbase.ttypes中定义 15 from hbase.ttypes import * 16 17 # Make socket 18 #此处可以修改地址和端口 19 transport = TSocket.TSocket('localhost', 19090) 20 # Buffering is critical. Raw sockets are very slow 21 # 还可以用TFramedTransport,也是高效传输方式 22 #transport = TTransport.TBufferedTransport(transport) 23 transport = TTransport.TFramedTransport(transport) 24 # Wrap in a protocol 25 #传输协议和传输过程是分离的,可以支持多协议 26 protocol = TBinaryProtocol.TBinaryProtocol(transport) 27 #客户端代表一个用户 28 client = Hbase.Client(protocol) 29 #打开连接 30 transport.open() 31 32 print client.getTableNames()
6.产看测试结果
python t.py
[user_growth@YZSJHL19-81 guoqiang.ma]$ python t.py
['atme', 'delete_friend', 'doing', 'doing2', 'fans_pages',
.........................................
.........................................
.........................................]
参考资料:
http://www.cnblogs.com/hitandrew/archive/2013/01/21/2870419.html
- 使用python访问hbase
- python访问Hbase
- python 访问hbase
- python 访问hbase
- python访问、操作hbase
- python 通过happybase 访问hbase
- python 访问hbase数据库代码,乱写
- HappyBase: 在 Python 中快速访问 HBase
- 使用Python通过Thrift接口访问HBase和Hive
- 在windows7上,使用python通过thrift访问hbase
- python解析hbase访问结果(TRowResult)的方式
- HBase--Java访问HBase示例
- python hbase
- HBase 之访问控制
- HBase 之访问控制
- hbase thrift 访问队列
- hive 访问hbase表
- thrift2访问hbase
- java语言概述
- poj1948(经典问题-二维背包 求面积最大三角形)
- JavaScript高级培训-自定义对象
- C++类中成员变量的初始化总结
- Javascript创建对象
- python 访问hbase
- JavaScript 创建自己的自定义对象
- 最优化javascript自定义对象
- javascript中的创建自定义对象的六种方式详解
- javascript创建自定义对象和prototype
- curl : 操作每一个文件时, 要成对执行 curl_easy_init 和 curl_easy_cleanup
- Sqoop导表在大并发下造成RegionServer的系统cpu过高
- iOS 设置titleview的宽度为屏幕宽
- 再谈JavaScript的用户自定义对象