apache phoenix简单部署测试

来源:互联网 发布:高速公路的数据 编辑:程序博客网 时间:2024/04/30 02:37

环境:

server 1:nn,hmaster,zkserver,

server2:nn backup,regionserver,zkserver

server3:regionserver,zkserver

habse 0.98.0

hadoop-2.3.0

zookeeper 3.4.5,

0.Apache phoenix是套在HBase外面的一层SQL皮,自己带JDBC driver,目标是通过SQL实现低延时的HBase数据查询,链接http://phoenix.apache.org

1.下载http://www.apache.org/dyn/closer.cgi/incubator/phoenix/ 最新的已编译的tarball,phoenix-4.0.0-incubating.tar.gz,在server1上解压到指定目录$PHOENIX_HOME=/app/phoenix

2.copy 已配置好的hbase-site.xml到$PHOENIX_HOME/bin下,复制$PHOENIX_HOME/common/phoenix-core-4.0.0-incubating.jar,$PHOENIX_HOME/hadoop-2/phoenix-core-4.0.0-incubating-tests.jar 包到$HBASE_HOME/lib下,如果有自己写的client,需要复制$PHOENIX_HOME/hadoop-2/phoenix-4.0.0-incubating-client.jar到对应classpath下(也支持hadoop1,复制对应目录下的jar包即可)

3.copy完成后重启hbase集群

4.确保hbase正常启动后,在server1上使用$PHOENIX_HOME/bin/sqlline.py server1,server2,server3 来启动命令行client,使用zk默认的2181端口和默认的/hbase目录,命令等效于$PHOENIX_HOME/bin/sqlline.py server1:2181,server2:2181,server3:2181:/hbase

5.启动时可能会报异常,实测不影响使用,原因没深究,如果有同学修正了这个错误欢迎指教。

Setting property: [isolation, TRANSACTION_READ_COMMITTED]issuing: !connect jdbc:phoenix:server3,server2 ,server1 none none org.apache.phoenix.jdbc.PhoenixDriverConnecting to jdbc:phoenix:server3,server214/07/18 15:28:24 WARN util.DynamicClassLoader: Failed to identify the fs of dir hdfs://server3:9009/hbase/lib, ignoredorg.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4        at org.apache.hadoop.ipc.Client.call(Client.java:1070)        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)        at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)        at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)        at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:104)        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:201)        at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)        at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:69)        at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83)        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:857)        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:662)        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)        at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:414)        at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:309)        at org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)        at org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:252)        at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1446)        at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:131)        at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:112)        at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4650)        at sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4701)        at sqlline.SqlLine$Commands.connect(SqlLine.java:3942)        at sqlline.SqlLine$Commands.connect(SqlLine.java:3851)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:606)        at sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2810)        at sqlline.SqlLine.dispatch(SqlLine.java:817)        at sqlline.SqlLine.initArgs(SqlLine.java:633)        at sqlline.SqlLine.begin(SqlLine.java:680)        at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)        at sqlline.SqlLine.main(SqlLine.java:424)Connected to: Phoenix (version 4.0)Driver: org.apache.phoenix.jdbc.PhoenixDriver (version 4.0)Autocommit status: trueTransaction isolation: TRANSACTION_READ_COMMITTEDBuilding list of tables and columns for tab-completion (set fastconnect to true to skip)...55/55 (100%) DoneDonesqlline version 1.1.2
0: jdbc:phoenix:gtx3,gtx2> !tables  #查看所有的表



6.创建表

0: jdbc:phoenix:server3,server2> create table test (mykey integer not null primary key, mycolumn varchar);No rows affected (1.293 seconds)0: jdbc:phoenix:server3,server2> upsert into test values (1,'Hello');1 row affected (0.046 seconds)0: jdbc:phoenix:server3,server2> upsert into test values (2,'World!');1 row affected (0.009 seconds)0: jdbc:phoenix:server3,server2> select * from test;+------------+------------+|   MYKEY    |  MYCOLUMN  |+------------+------------+| 1          | Hello      || 2          | World!     |+------------+------------+

7.使用已有的表,貌似在hbase中插入的数据,在这里字段值i西安是为null,具体原因还没来得及看,回头再研究下,也欢迎指教

hbase中已存在表t1,需要创建该表的视图

0: jdbc:phoenix:server3,server2> CREATE VIEW "t1" ( pk VARCHAR PRIMARY KEY, "cf1".val VARCHAR )0: jdbc:phoenix:server3,server2> select * from "t1";+------------+------------+|     PK     |    VAL     |+------------+------------+| r1         | null       || r13        | null       || r2         | null       |+------------+------------+


0 0