访问Hbase,待更新方法一
来源:互联网 发布:图像清晰度检测算法 编辑:程序博客网 时间:2024/06/05 14:09
访问Hbase的三种方法:
方法一:HbaseTemlate工具类
(1)配置文件:src\main\resources\spring-hbase.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:hdp="http://www.springframework.org/schema/hadoop"
xmlns:beans="http://www.springframework.org/schema/beans"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">
<!-- 配置zookeeper的信息,远程连接hbase时使用 -->
<hdp:configuration resources="classpath:/hbase-site.xml" />
<hdp:hbase-configuration configuration-ref="hadoopConfiguration" />
<!-- 配置HbaseTemplate -->
<bean id="htemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate">
<property name="configuration" ref="hbaseConfiguration">
</property>
<property name="encoding" value="UTF-8"></property>
</bean>
</beans>
(2)配置文件:src\main\resources\hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<!-- zookeeper集群的URL配置 -->
<property>
<name>hbase.zookeeper.quorum</name>
<value>namenode1-sit.cnsuning.com,namenode2-sit.cnsuning.com,slave01-sit.cnsuning.com</value>
</property>
<!-- 客户端连接端口 -->
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2015</value>
</property>
<!-- 重试等待时间 -->
<property>
<name>hbase.client.pause</name>
<value>50</value>
</property>
<!-- 失败时重试次数 -->
<property>
<name>hbase.client.retries.number</name>
<value>3</value>
</property>
<!-- RPC请求的超时时间。如果某次RPC时间超过该值,客户端就会主动关闭socket -->
<property>
<name>hbase.rpc.timeout</name>
<value>2000</value>
</property>
<!-- HBase客户端发起一次数据操作直至得到响应之间总的超时时间 -->
<property>
<name>hbase.client.operation.timeout</name>
<value>3000</value>
</property>
<!-- 该参数是表示HBase客户端发起一次scan操作的rpc调用至得到响应之间总的超时时间 -->
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>10000</value>
</property>
</configuration>
(2)java程序:
static private ApplicationContext hbaseContext = new ClassPathXmlApplicationContext("classpath:spring-hbase.xml");
static private BeanFactory factory = (BeanFactory) hbaseContext;
static private HbaseTemplate htemplate = (HbaseTemplate) factory.getBean("htemplate");
static private GetRow getRow = new GetRow();//回调函数
String tableName = "ns_sousuo:tdm_empower_gds_init_d";
String rowKey = "20160919_000000_ka_000000300_000000000";
System.setProperty("HADOOP_USER_NAME", "sousuo");
String hbaseRes = "";
try{
hbaseRes = htemplate.get(tableName, rowKey, getRow);
}catch(Exception e){
logger.error(e.getMessage(), e);
}
其中回调函数GetRow如下
public String mapRow(Result result, int rowNum) throws Exception {
String hbaseRes = "";
List<Cell> ceList = result.listCells();
if(ceList!=null&&ceList.size()>0){
for(Cell cell:ceList){
hbaseRes += Bytes.toString(cell.getFamilyArray(),cell.getFamilyOffset(),cell.getFamilyLength())+
"_"+Bytes.toString(cell.getQualifierArray(),cell.getQualifierOffset(),cell.getQualifierLength())
+"="+Bytes.toString( cell.getValueArray(), cell.getValueOffset(), cell.getValueLength())+" ";
}
}
return hbaseRes;
}
方法二:Hbase线程池访问:
Map<String,String> numAndPort = getNumAndPort();
Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.quorum", numAndPort.get("hbase.zookeeper.quorum"));
configuration.set("hbase.zookeeper.property.clientPort", numAndPort.get("hbase.zookeeper.property.clientPort"));
HConnection connection = null;
HTableInterface table = null;
try {
configuration.set("hbase.client.pause", "50"); //重试等待时间
configuration.set("hbase.client.retries.number", "3"); //失败时重试次数
configuration.set("hbase.rpc.timeout", "2000"); //RPC请求的超时时间。如果某次RPC时间超过该值,客户端就会主动关闭socket
configuration.set("hbase.client.operation.timeout", "3000"); //HBase客户端发起一次数据操作直至得到响应之间总的超时时间
configuration.set("hbase.client.scanner.timeout.period", "10000");//该参数是表示HBase客户端发起一次scan操作的rpc调用至得到响应之间总的超时时间
connection = HConnectionManager.createConnection(configuration);
table = connection.getTable(tableName);
hbaseRes = getHbaseRow(table, tableName, rowKey);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}finally {
table.close();
connection.close();
}
getHbaseRow方法:
public static String getHbaseRow(HTableInterface table, String tableName, String row) throws Exception {
Get get = new Get(Bytes.toBytes(row));
Result result = table.get(get);
String returnResult = "";
// 输出结果,raw方法返回所有keyvalue数组
for (KeyValue rowKV : result.raw()) {
System.out.print("行名:" + new String(rowKV.getRow()) + " ");
System.out.print("时间戳:" + rowKV.getTimestamp() + " ");
System.out.print("列族名:" + new String(rowKV.getFamily()) + " ");
System.out.print("列名:" + new String(rowKV.getQualifier()) + " ");
System.out.println("值:" + new String(rowKV.getValue()));
returnResult = new String(rowKV.getValue());
}
return returnResult;
}
方法三:将Hbase的配置写到一般的xml,然后用java获取配置文件中的内容:
(1)配置文件:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property name="hbase.zookeeper.quorum" value= "namenode1-sit.cnsuning.com,namenode2-sit.cnsuning.com,slave01-sit.cnsuning.com">
</property>
<property name="hbase.zookeeper.property.clientPort" value="2015">
</property>
</configuration>
(2)java程序:
Map<String,String> numAndPort = getNumAndPort();
Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.quorum", numAndPort.get("hbase.zookeeper.quorum"));
configuration.set("hbase.zookeeper.property.clientPort", numAndPort.get("hbase.zookeeper.property.clientPort"));
HBaseAdmin hAdmin = new HBaseAdmin(configuration);
boolean aa = hAdmin.tableExists("ns_sousuo:tdm_empower_gds_init_d");
if(aa)
{
System.out.println("表存在!!!");
HTable table = new HTable(configuration, "ns_sousuo:tdm_empower_gds_init_d");
Scan scan = new Scan();
ResultScanner results = table.getScanner(scan);
// 输出结果
for (Result result1 : results) {
for (KeyValue rowKV : result1.raw()) {
System.out.print("行名:" + new String(rowKV.getRow()) + " ");
System.out.print("时间戳:" + rowKV.getTimestamp() + " ");
System.out.print("列族名:" + new String(rowKV.getFamily()) + " ");
System.out
.print("列名:" + new String(rowKV.getQualifier()) + " ");
System.out.println("值:" + new String(rowKV.getValue()));
}
}
}
else
{
System.out.println("表不存在!!!");
}
方法一:HbaseTemlate工具类
(1)配置文件:src\main\resources\spring-hbase.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:hdp="http://www.springframework.org/schema/hadoop"
xmlns:beans="http://www.springframework.org/schema/beans"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">
<!-- 配置zookeeper的信息,远程连接hbase时使用 -->
<hdp:configuration resources="classpath:/hbase-site.xml" />
<hdp:hbase-configuration configuration-ref="hadoopConfiguration" />
<!-- 配置HbaseTemplate -->
<bean id="htemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate">
<property name="configuration" ref="hbaseConfiguration">
</property>
<property name="encoding" value="UTF-8"></property>
</bean>
</beans>
(2)配置文件:src\main\resources\hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<!-- zookeeper集群的URL配置 -->
<property>
<name>hbase.zookeeper.quorum</name>
<value>namenode1-sit.cnsuning.com,namenode2-sit.cnsuning.com,slave01-sit.cnsuning.com</value>
</property>
<!-- 客户端连接端口 -->
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2015</value>
</property>
<!-- 重试等待时间 -->
<property>
<name>hbase.client.pause</name>
<value>50</value>
</property>
<!-- 失败时重试次数 -->
<property>
<name>hbase.client.retries.number</name>
<value>3</value>
</property>
<!-- RPC请求的超时时间。如果某次RPC时间超过该值,客户端就会主动关闭socket -->
<property>
<name>hbase.rpc.timeout</name>
<value>2000</value>
</property>
<!-- HBase客户端发起一次数据操作直至得到响应之间总的超时时间 -->
<property>
<name>hbase.client.operation.timeout</name>
<value>3000</value>
</property>
<!-- 该参数是表示HBase客户端发起一次scan操作的rpc调用至得到响应之间总的超时时间 -->
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>10000</value>
</property>
</configuration>
(2)java程序:
static private ApplicationContext hbaseContext = new ClassPathXmlApplicationContext("classpath:spring-hbase.xml");
static private BeanFactory factory = (BeanFactory) hbaseContext;
static private HbaseTemplate htemplate = (HbaseTemplate) factory.getBean("htemplate");
static private GetRow getRow = new GetRow();//回调函数
String tableName = "ns_sousuo:tdm_empower_gds_init_d";
String rowKey = "20160919_000000_ka_000000300_000000000";
System.setProperty("HADOOP_USER_NAME", "sousuo");
String hbaseRes = "";
try{
hbaseRes = htemplate.get(tableName, rowKey, getRow);
}catch(Exception e){
logger.error(e.getMessage(), e);
}
其中回调函数GetRow如下
public String mapRow(Result result, int rowNum) throws Exception {
String hbaseRes = "";
List<Cell> ceList = result.listCells();
if(ceList!=null&&ceList.size()>0){
for(Cell cell:ceList){
hbaseRes += Bytes.toString(cell.getFamilyArray(),cell.getFamilyOffset(),cell.getFamilyLength())+
"_"+Bytes.toString(cell.getQualifierArray(),cell.getQualifierOffset(),cell.getQualifierLength())
+"="+Bytes.toString( cell.getValueArray(), cell.getValueOffset(), cell.getValueLength())+" ";
}
}
return hbaseRes;
}
方法二:Hbase线程池访问:
Map<String,String> numAndPort = getNumAndPort();
Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.quorum", numAndPort.get("hbase.zookeeper.quorum"));
configuration.set("hbase.zookeeper.property.clientPort", numAndPort.get("hbase.zookeeper.property.clientPort"));
HConnection connection = null;
HTableInterface table = null;
try {
configuration.set("hbase.client.pause", "50"); //重试等待时间
configuration.set("hbase.client.retries.number", "3"); //失败时重试次数
configuration.set("hbase.rpc.timeout", "2000"); //RPC请求的超时时间。如果某次RPC时间超过该值,客户端就会主动关闭socket
configuration.set("hbase.client.operation.timeout", "3000"); //HBase客户端发起一次数据操作直至得到响应之间总的超时时间
configuration.set("hbase.client.scanner.timeout.period", "10000");//该参数是表示HBase客户端发起一次scan操作的rpc调用至得到响应之间总的超时时间
connection = HConnectionManager.createConnection(configuration);
table = connection.getTable(tableName);
hbaseRes = getHbaseRow(table, tableName, rowKey);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}finally {
table.close();
connection.close();
}
getHbaseRow方法:
public static String getHbaseRow(HTableInterface table, String tableName, String row) throws Exception {
Get get = new Get(Bytes.toBytes(row));
Result result = table.get(get);
String returnResult = "";
// 输出结果,raw方法返回所有keyvalue数组
for (KeyValue rowKV : result.raw()) {
System.out.print("行名:" + new String(rowKV.getRow()) + " ");
System.out.print("时间戳:" + rowKV.getTimestamp() + " ");
System.out.print("列族名:" + new String(rowKV.getFamily()) + " ");
System.out.print("列名:" + new String(rowKV.getQualifier()) + " ");
System.out.println("值:" + new String(rowKV.getValue()));
returnResult = new String(rowKV.getValue());
}
return returnResult;
}
方法三:将Hbase的配置写到一般的xml,然后用java获取配置文件中的内容:
(1)配置文件:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property name="hbase.zookeeper.quorum" value= "namenode1-sit.cnsuning.com,namenode2-sit.cnsuning.com,slave01-sit.cnsuning.com">
</property>
<property name="hbase.zookeeper.property.clientPort" value="2015">
</property>
</configuration>
(2)java程序:
Map<String,String> numAndPort = getNumAndPort();
Configuration configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.quorum", numAndPort.get("hbase.zookeeper.quorum"));
configuration.set("hbase.zookeeper.property.clientPort", numAndPort.get("hbase.zookeeper.property.clientPort"));
HBaseAdmin hAdmin = new HBaseAdmin(configuration);
boolean aa = hAdmin.tableExists("ns_sousuo:tdm_empower_gds_init_d");
if(aa)
{
System.out.println("表存在!!!");
HTable table = new HTable(configuration, "ns_sousuo:tdm_empower_gds_init_d");
Scan scan = new Scan();
ResultScanner results = table.getScanner(scan);
// 输出结果
for (Result result1 : results) {
for (KeyValue rowKV : result1.raw()) {
System.out.print("行名:" + new String(rowKV.getRow()) + " ");
System.out.print("时间戳:" + rowKV.getTimestamp() + " ");
System.out.print("列族名:" + new String(rowKV.getFamily()) + " ");
System.out
.print("列名:" + new String(rowKV.getQualifier()) + " ");
System.out.println("值:" + new String(rowKV.getValue()));
}
}
}
else
{
System.out.println("表不存在!!!");
}
阅读全文
0 0
- 访问Hbase,待更新方法一
- 访问Hbase,待更新方法一
- Android_02_文件访问权限(待更新)
- Hbase JavaAPi介绍和使用示例(待更新)
- Python: 爬虫方法小结[待更新!!!]
- 进制间的转换方法--待更新
- 待更新
- 待更新
- 待解决hbase异常
- 非集群client访问HBase方法
- 非集群client访问HBase方法
- <PY>Python扩展包安装方法(待更新)
- shell中变量自增的实现方法 ----待更新
- JS数组练习可能用到的方法(待更新)
- HBase学习(一)hbase安装(单机模式)和javaapi客户端访问hbase例子
- Hbase 基础 (待完善)
- 批处理网站【待更新】
- 排序算法(待更新)
- path路径大解析
- 关于字符编码,你所需要知道的(ASCII,Unicode,Utf-8,GB2312…)
- 论文需要的几张图片
- idea主题设置
- 如何避免点击事件被执行多次
- 访问Hbase,待更新方法一
- PAT (Advanced Level) Practise 1038 Recover the Smallest Number (30)
- 尝试加载oracle客户端时引发BadImageFormatException.如果在安装32位Oracle客户端组件的情况下以64位模式运行
- 替换字符串中的空格为$$$。要求时间复杂度为O(N)
- CSDN博客积分规则和获取积分方法
- HTML&CSS精选笔记_浮动与定位
- UML之行为图
- 51Nod 1091 线段的重叠
- freebsd 与 nginx 与 openssl编译 http2