Thrift

来源：互联网发布：旅游对大数据需求编辑：程序博客网时间：2024/06/07 02:40

1.IDL （interface description language）

1.命名空间

namespace java com.zhp 适用java语言
namespace * com.zhp 适用于所有语言

2.包含外部文件

include "chapter3.thrift"

引用包含文件的内容时使用 前缀.xxx。例如
list<string> getList(1:set<i32> ids) throws (1:MyError2 myerror,2:chapter3.MyError merror2);
3.自定义类型以简化类型定义
typedef i32 inttypedef map<string,i32> myMap
exception MyError{    1:int error_code,    2:string error_desc}
使用int 代替了 i32
4.定义常量
简单类型
const double PI=3.1415926;
集合类型
const map<i32,string> CITYS={1:"shanghang",2:"beijing",3:"hangzhou"};const list<string> lan=["java","php","c++"];
类实例struct city{    1:string name;    2:i32 pop;}const city SHANG_HAI ={"name":"shang_hai","pop":1000};
5.定义实体类
struct Teacher{    1: required bool zhp,    2: optional byte bete,    3: optional i16  by16,    4: required i32 by32=0,    5: required i64 by64,    6: optional double d=0.1,    7: optional string str="zhp"}
我们看生成的代码（thrift --gen java demo.thrift 生成代码）
1.读取序列化数据，我们看到是通过IDL中定义的字段顺序来设置字段的。我们有设置required 最后有两个校验。
public void read(org.apache.thrift.protocol.TProtocol iprot, Teacher struct) throws org.apache.thrift.TException {  org.apache.thrift.protocol.TField schemeField;  iprot.readStructBegin();  while (true)  {    schemeField = iprot.readFieldBegin();    if (schemeField.type == org.apache.thrift.protocol.TType.STOP) {       break;    }    switch (schemeField.id) {      case 1: // ZHP        if (schemeField.type == org.apache.thrift.protocol.TType.BOOL) {          struct.zhp = iprot.readBool();          struct.setZhpIsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      case 2: // BETE        if (schemeField.type == org.apache.thrift.protocol.TType.BYTE) {          struct.bete = iprot.readByte();          struct.setBeteIsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      case 3: // BY16        if (schemeField.type == org.apache.thrift.protocol.TType.I16) {          struct.by16 = iprot.readI16();          struct.setBy16IsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      case 4: // BY32        if (schemeField.type == org.apache.thrift.protocol.TType.I32) {          struct.by32 = iprot.readI32();          struct.setBy32IsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      case 5: // BY64        if (schemeField.type == org.apache.thrift.protocol.TType.I64) {          struct.by64 = iprot.readI64();          struct.setBy64IsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      case 6: // D        if (schemeField.type == org.apache.thrift.protocol.TType.DOUBLE) {          struct.d = iprot.readDouble();          struct.setDIsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      case 7: // STR        if (schemeField.type == org.apache.thrift.protocol.TType.STRING) {          struct.str = iprot.readString();          struct.setStrIsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }        break;      default:        org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);    }    iprot.readFieldEnd();  }  iprot.readStructEnd();  // check for required fields of primitive type, which can't be checked in the validate method  if (!struct.isSetZhp()) {    throw new org.apache.thrift.protocol.TProtocolException("Required field 'zhp' was not found in serialized data! Struct: " + toString());  }  if (!struct.isSetBy32()) {    throw new org.apache.thrift.protocol.TProtocolException("Required field 'by32' was not found in serialized data! Struct: " + toString());  }  if (!struct.isSetBy64()) {    throw new org.apache.thrift.protocol.TProtocolException("Required field 'by64' was not found in serialized data! Struct: " + toString());  }  struct.validate();}

我们看到struct.validate(); 它是一个空方法需要我们自己去写校验。

public void validate() throws org.apache.thrift.TException {  // check for required fields  // alas, we cannot check 'zhp' because it's a primitive and you chose the non-beans generator.  // alas, we cannot check 'by32' because it's a primitive and you chose the non-beans generator.  // alas, we cannot check 'by64' because it's a primitive and you chose the non-beans generator.  // check for sub-struct validity}

2.我们再看序列化数据的时候，先调用校验接口，然后先写入类型，再写入字段。

 public void write(org.apache.thrift.protocol.TProtocol oprot, Teacher struct) throws org.apache.thrift.TException {    struct.validate();    oprot.writeStructBegin(STRUCT_DESC);    oprot.writeFieldBegin(ZHP_FIELD_DESC);    oprot.writeBool(struct.zhp);    oprot.writeFieldEnd();    if (struct.isSetBete()) {      oprot.writeFieldBegin(BETE_FIELD_DESC);      oprot.writeByte(struct.bete);      oprot.writeFieldEnd();    }    if (struct.isSetBy16()) {      oprot.writeFieldBegin(BY16_FIELD_DESC);      oprot.writeI16(struct.by16);      oprot.writeFieldEnd();    }    oprot.writeFieldBegin(BY32_FIELD_DESC);    oprot.writeI32(struct.by32);    oprot.writeFieldEnd();    oprot.writeFieldBegin(BY64_FIELD_DESC);    oprot.writeI64(struct.by64);    oprot.writeFieldEnd();    if (struct.isSetD()) {      oprot.writeFieldBegin(D_FIELD_DESC);      oprot.writeDouble(struct.d);      oprot.writeFieldEnd();    }    if (struct.str != null) {      if (struct.isSetStr()) {        oprot.writeFieldBegin(STR_FIELD_DESC);        oprot.writeString(struct.str);        oprot.writeFieldEnd();      }    }    oprot.writeFieldStop();    oprot.writeStructEnd();  }}

private static final org.apache.thrift.protocol.TStruct STRUCT_DESC = new org.apache.thrift.protocol.TStruct("Teacher");private static final org.apache.thrift.protocol.TField ZHP_FIELD_DESC = new org.apache.thrift.protocol.TField("zhp", org.apache.thrift.protocol.TType.BOOL, (short)1);

我们看写入字段的时候包括字段名，类型，和顺序。而read的时候就是根据这个顺序值进行设置的。

case 1: // ZHP        if (schemeField.type == org.apache.thrift.protocol.TType.BOOL) {          struct.zhp = iprot.readBool();          struct.setZhpIsSet(true);        } else {           org.apache.thrift.protocol.TProtocolUtil.skip(iprot, schemeField.type);        }   break;

得出结论，只要字段的顺序和类型保持不变就能正确序列化和反序列化。

5.定义枚举

enum Position{  CEO,CFO,PM,PD=9}

其他地方引用这个枚举

struct Stuedent{    1: string name,    2: Teacher teacher,    3: list<i32> score,    4: set<string> names,    5: map<string,i32> maps,    6: Position po}

最后枚举生成代码如下，PD设置了9，其他默认从0开始递增。

public enum Position implements org.apache.thrift.TEnum {  CEO(0),  CFO(1),  PM(2),  PD(9);  private final int value;  private Position(int value) {    this.value = value;  }

6.定义接口

exception MyError2{    1:i32 code,    2:string msg}service ZhpService{    void say(1:Teacher zhpS);    void sayNum(1:StuTea st);    set<i32> getNums(1:list<string> alist);    void testEnum(1:Position p);    list<string> getList(1:set<i32> ids) throws (1:MyError2 myerror,2:chapter3.MyError merror2);    //单向 不等待结果的返回 非阻塞    oneway void noWaiteResult(1:string id);}

7.接口继承

service PeopleDirectory{    oneway void log(1:string  message);    void reloadDatabase();}service EmployeeDirectory extends PeopleDirectory{    Stuedent findStu(1:i32 int);}

8.oneway

”oneway”标识符表示client发出请求后不必等待回复（非阻塞）直接进行下面的操作，
”oneway”方法的返回值必须是void

2.Thrift network stack

  +-------------------------------------------+  | Server                                    |  | (single-threaded, event-driven etc)       |  +-------------------------------------------+  | Processor                                 |  | (compiler generated)                      |  +-------------------------------------------+  | Protocol                                  |  | (JSON, compact etc)                       |  +-------------------------------------------+  | Transport                                 |  | (raw TCP, HTTP etc)                       |  +-------------------------------------------+

2.1 Server的几种实现

2.1.1 TSimpleServer

服务端代码

//简单测试模型//请求的处理器（接口的实现）TProcessor processor = new HelloWorldService.Processor<>(new HelloWorldImpl());//服务端socket传输TServerTransport serverSocket = new TServerSocket(8089);
TServer.Args targs = new TServer.Args(serverSocket);targs.processor(processor);//传输的数据是二进制形式targs.protocolFactory(new TBinaryProtocol.Factory());TServer server = new TSimpleServer(targs);System.out.println("服务启动");server.serve();

客户端代码

//传输层 socket tcpTTransport tTransport = new TSocket("localhost", 8089, 2000);//数据以二进制形式传输TProtocol protocol = new TBinaryProtocol(tTransport);HelloWorldService.Client client = new HelloWorldService.Client(protocol);//打开sockettTransport.open();String zhp = client.sayHello("zhp");System.out.println(zhp);

2.1.2 TThreadPoolServer

服务端代码

//线程池服务模型，使用标准的阻塞式IO，预先创建一组线程处理请求//请求的处理器（接口的实现）TProcessor processor = new HelloWorldService.Processor<>(new HelloWorldImpl());//服务端socket传输TServerTransport serverSocket = new TServerSocket(8089);TThreadPoolServer.Args targs = new TThreadPoolServer.Args(serverSocket);targs.processor(processor);//传输的数据是二进制形式targs.protocolFactory(new TBinaryProtocol.Factory());TServer server = new TThreadPoolServer(targs);System.out.println("服务启动");server.serve();

客户端代码和TSimpleServer一样。

注意：默认线程池队列是无容量的，线程数量上限是int类型最大值。在实际使用过程中一定要根据业务设置最大线程数量。

public int minWorkerThreads = 5;public int maxWorkerThreads = Integer.MAX_VALUE;

private static ExecutorService createDefaultExecutorService(Args args) {  SynchronousQueue<Runnable> executorQueue =    new SynchronousQueue<Runnable>();  return new ThreadPoolExecutor(args.minWorkerThreads,                                args.maxWorkerThreads,                                args.stopTimeoutVal,                                TimeUnit.SECONDS,                                executorQueue);}

2.1.3 TNonblockingServer

服务端代码

//使用非阻塞式IO，服务端和客户端需要指定 TFramedTransport 数据传输的方式。//指定非阻塞服务socket//绑定地址和端口 创建ServerSocket对象TNonblockingServerTransport serverSocket = new TNonblockingServerSocket(8089);//服务实现TProcessor processor = new HelloWorldService.Processor<>(new HelloWorldImpl());//构建非阻塞服务参数TNonblockingServer.Args targs = new TNonblockingServer.Args(serverSocket);targs.processor(processor);//数据以帧的方式传输targs.transportFactory(new TFramedTransport.Factory());//数据压缩targs.protocolFactory(new TCompactProtocol.Factory());//服务启动TServer server = new TNonblockingServer(targs);server.serve();

客户端代码

//使用非阻塞式IO，服务端和客户端需要指定 TFramedTransport 数据传输的方式。TTransport tTransport = new TSocket("localhost", 8089, 2000);TFramedTransport tFramedTransport = new TFramedTransport(tTransport);TProtocol protocol = new TCompactProtocol(tFramedTransport);HelloWorldService.Client client = new HelloWorldService.Client(protocol);tFramedTransport.open();String zhp = client.sayHello("zhp");System.out.println(zhp);

注意：只有一个线程在处理。如果读取完数据后进行业务处理速度慢，则会阻塞其他处理。

2.1.4 THsHaServer

服务端代码

//非阻塞IO，半同步半异步的服务模型（读完数据后异步处理业务）TNonblockingServerTransport serverSocket = new TNonblockingServerSocket(8089);//服务实现TProcessor processor = new HelloWorldService.Processor<>(new HelloWorldImpl());THsHaServer.Args targs = new THsHaServer.Args(serverSocket);targs.processor(processor);targs.transportFactory(new TFramedTransport.Factory());targs.protocolFactory(new TCompactProtocol.Factory());TServer server = new THsHaServer(targs);server.serve();

客户端代码和 2.1.3 的一样。

我们看THsHaServer 是继承于TNonblockingServer

public class THsHaServer extends TNonblockingServer

只是重写了requestInvoke将同步调用改为异步调用。其他的和TNonblockingServer一样。

@Overrideprotected boolean requestInvoke(FrameBuffer frameBuffer) {  try {    Runnable invocation = getRunnable(frameBuffer);    invoker.execute(invocation);    return true;  } catch (RejectedExecutionException rx) {    LOGGER.warn("ExecutorService rejected execution!", rx);    return false;  }}

使用线程池异步进行业务处理。

注意默认线程池

protected static ExecutorService createInvokerPool(Args options) {  int minWorkerThreads = options.minWorkerThreads;  int maxWorkerThreads = options.maxWorkerThreads;  int stopTimeoutVal = options.stopTimeoutVal;  TimeUnit stopTimeoutUnit = options.stopTimeoutUnit;  LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>();  ExecutorService invoker = new ThreadPoolExecutor(minWorkerThreads,    maxWorkerThreads, stopTimeoutVal, stopTimeoutUnit, queue);  return invoker;}

是无界阻塞队列。使用的时候最后自定义线程池，防止内存被撑爆。

2.1.5 TThreadedSelectorServer

服务端代码

//非阻塞IO，半同步半异步服务模型TNonblockingServerTransport serverTransport = new TNonblockingServerSocket(8089);//服务实现TProcessor processor = new HelloWorldService.Processor<>(new HelloWorldImpl());TThreadedSelectorServer.Args targs = new TThreadedSelectorServer.Args(serverTransport);targs.processor(processor);targs.transportFactory(new TFramedTransport.Factory());targs.protocolFactory(new TCompactProtocol.Factory());TServer server = new TThreadedSelectorServer(targs);server.serve();

客户端代码和 2.1.3 一样

默认的工作线程池是5个线程

/** * Helper to create the invoker if one is not specified */protected static ExecutorService createDefaultExecutor(Args options) {  return (options.workerThreads > 0) ? Executors.newFixedThreadPool(options.workerThreads) : null;}

/** * The size of the executor service (if none is specified) that will handle * invocations. This may be set to 0, in which case invocations will be * handled directly on the selector threads (as is in TNonblockingServer) */private int workerThreads = 5;

selectorThread是2个线程

/** The number of threads for selecting on already-accepted connections */public int selectorThreads = 2;

2.1.6 异步客户端

异步客户端只适用于非阻塞IO。代码如下

TAsyncClientManager clientManager = new TAsyncClientManager();TNonblockingTransport tTransport = new TNonblockingSocket("localhost", 8089, 200);TProtocolFactory tProtocolFactory = new TCompactProtocol.Factory();HelloWorldService.AsyncClient client = new HelloWorldService.AsyncClient(tProtocolFactory, clientManager, tTransport);client.sayHello("zhp", new AsyncMethodCallback<String>() {    @Override    public void onComplete(String s) {        System.out.println(s);    }    @Override    public void onError(Exception e) {        e.printStackTrace();    }});

阅读全文

0 0