spark SQL Running the Thrift JDBC/ODBC server

来源：互联网发布：九维网络登录编辑：程序博客网时间：2024/05/01 02:01

Running the Thrift JDBC/ODBC server

1:运行

./sbin/start-thriftserver.sh --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.server2.thrift.bind.host=feng02 --master spark://feng02:7077 --driver-class-path /home/jifeng/hadoop/spark-1.2.0-bin-2.4.1/lib/mysql-connector-java-5.1.32-bin.jar --executor-memory 1g

端口:10000

服务器:feng02

spark master:spark://feng02:7077

driver-class-path:mysql驱动包(hive配置的)

[jifeng@feng02 spark-1.2.0-bin-2.4.1]$ ./sbin/start-thriftserver.sh  --hiveconf hive.server2.thrift.port=10000  --hiveconf hive.server2.thrift.bind.host=feng02 --master spark://feng02:7077 --driver-class-path /home/jifeng/hadoop/spark-1.2.0-bin-2.4.1/lib/mysql-connector-java-5.1.32-bin.jarstarting org.apache.spark.sql.hive.thriftserver.HiveThriftServer2, logging to /home/jifeng/hadoop/spark-1.2.0-bin-2.4.1/sbin/../logs/spark-jifeng-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-feng02.out

2:运行beeline

Now you can use beeline to test the Thrift JDBC/ODBC server:

./bin/beeline

[jifeng@feng02 spark-1.2.0-bin-2.4.1]$ ./bin/beelineSpark assembly has been built with Hive, including Datanucleus jars on classpathBeeline version ??? by Apache Hive

3:连接server

参考：https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-BeelineExample

beeline> !connect jdbc:hive2://feng02:10000 jifeng jifeng org.apache.hive.jdbc.HiveDriverConnecting to jdbc:hive2://feng02:10000log4j:WARN No appenders could be found for logger (org.apache.thrift.transport.TSaslTransport).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.Connected to: Spark SQL (version 1.2.0)Driver: null (version null)Transaction isolation: TRANSACTION_REPEATABLE_READ

4：查询

0: jdbc:hive2://feng02:10000> show tables;                                                                  +----------------+|     result     |+----------------+| course         || hbase_table_1  || pokes          || student        |+----------------+4 rows selected (2.723 seconds)

0: jdbc:hive2://feng02:10000> select * from student;+-----+----------+------+| id  |   name   | age  |+-----+----------+------+| 1   | nick     | 24   || 2   | doping   | 25   || 3   | caizhi   | 26   || 4   | liaozhi  | 27   || 5   | wind     | 30   |+-----+----------+------+5 rows selected (10.554 seconds)0: jdbc:hive2://feng02:10000> select a.*,b.* from student a  join course b where a.id=b.id ;+-----+----------+------+-----+-----+-----+-----+-----+| id  |   name   | age  | id  | c1  | c2  | c3  | c4  |+-----+----------+------+-----+-----+-----+-----+-----+| 1   | nick     | 24   | 1   | 英语  | 中文  | 法文  | 日文  || 2   | doping   | 25   | 2   | 中文  | 法文  |     |     || 3   | caizhi   | 26   | 3   | 中文  | 法文  | 日文  |     || 4   | liaozhi  | 27   | 4   | 中文  | 法文  | 拉丁  |     || 5   | wind     | 30   | 5   | 中文  | 法文  | 德文  |     |+-----+----------+------+-----+-----+-----+-----+-----+5 rows selected (2.33 seconds)

4:Java JDBC连接

package demo.test;import java.sql.*;public class Pretest {    public static void main( String args[] )        throws SQLException , ClassNotFoundException {        String jdbcdriver="org.apache.hive.jdbc.HiveDriver";        String jdbcurl="jdbc:hive2://feng02:10000";        String username="scott";        String password="tiger";        Class.forName(jdbcdriver);        Connection c = DriverManager.getConnection(jdbcurl,username,password);         Statement st = c.createStatement();                print( "num should be 1 " , st.executeQuery("select * from student"));        // TODO indexing    } static void print( String name , ResultSet res )        throws SQLException {        System.out.println( name);        ResultSetMetaData meta=res.getMetaData();                        //System.out.println( "\t"+res.getRow()+"条记录");                Stringstr="";                for(int i=1;i<=meta.getColumnCount();i++){                str+=meta.getColumnName(i)+"   ";                //System.out.println( meta.getColumnName(i)+"   ");                }                System.out.println("\t"+str);                str="";        while ( res.next() ){        for(int i=1;i<=meta.getColumnCount();i++){        str+= res.getString(i)+"   ";        }         System.out.println("\t"+str);        str="";        }    }    }

上面是运行参数

结果显示：

num should be 1 id   name   age   1   nick   24   2   doping   25   3   caizhi   26   4   liaozhi   27   5   wind   30

0 0