spark-sql(二)---ThriftServer模式

ThriftServer

ThriftServer和和hiveserver2类似，提供了jdbc/odbc的接口，spark-sql以这种模式启动后可以以服务的形式提供SQL查询。
客户端可以用jdbc直接访问spark-sql，用sql进行交互。

启动

安装配置接前面链接
http://blog.csdn.net/dante_003/article/details/78652534
配置文件用的上个连接 conf/hive-site.xml

sbin/start-thriftserver.sh --master yarn \--driver-cores 2 --driver-memory 2g \--executor-cores 13 --executor-memory 16g \--num-executors 3

beeline

注意：一定要用spark文件夹中的beeline，不用使用hive文件夹中的beeline

[root@ht05 bin]# bin/beeline -u jdbc:hive2://localhost:10000Connecting to jdbc:hive2://localhost:1000017/11/28 17:43:40 INFO jdbc.Utils: Supplied authorities: localhost:1000017/11/28 17:43:40 INFO jdbc.Utils: Resolved authority: localhost:1000017/11/28 17:43:40 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10000Connected to: Spark SQL (version 2.2.0)Driver: Hive JDBC (version 1.2.1.spark2)Transaction isolation: TRANSACTION_REPEATABLE_READBeeline version 1.2.1.spark2 by Apache Hive0: jdbc:hive2://localhost:10000> show tables;+-----------+---------------+--------------+--+| database  |   tableName   | isTemporary  |+-----------+---------------+--------------+--+| default   | test_orc      | false        || default   | test_orc_t    | false        || default   | test_phoenix  | false        || default   | test_txt      | false        || default   | test_txt1     | false        |+-----------+---------------+--------------+--+5 rows selected (0.587 seconds)0: jdbc:hive2://localhost:10000> select count(*) from test_txt1;+--------------+--+|   count(1)   |+--------------+--+| 12818207296  |+--------------+--+1 row selected (262.646 seconds)

java jdbc连接

取出jdbc驱动，位置在spark/jars/hive-jdbc-1.2.1.spark2.jar
TestSparksql.java

import java.sql.Connection;import java.sql.DriverManager;import java.sql.ResultSet;import java.sql.SQLException;import java.sql.Statement;public class TestSparksql {    private static String driverName="org.apache.hive.jdbc.HiveDriver";    public static void main(String[] args) throws ClassNotFoundException, SQLException {        Class.forName(driverName);        Connection con=DriverManager.getConnection("jdbc:hive2://192.168.2.5:10000/default", "", "");        Statement statement = con.createStatement();        String sql="select * from test_txt1 limit 10";        ResultSet executeQuery = statement.executeQuery(sql);        while(executeQuery.next()){            String string = executeQuery.getString(1);            System.out.println(string);        }        statement.close();        con.close();    }}