Avro：RPC例子

来源：互联网发布：ae mac 渲染快捷键编辑：程序博客网时间：2024/04/29 07:12

Apache Avro除了提供一种数据序列化机制以外，还提供了一套RPC机制。本文基于Avro协议，实现一个Java语言编写的RPC Server，以及一个Python语言编写的客户端。

定义一个RPC协议

Acro的RPC协议也是使用JSON来定义，通常以.avpr结尾。这个例子中我们实现一个简单的邮件发送服务，该服务定义如下：

{    "namespace": "me.lin.avro.rpc",    "protocol": "Mail",    "types": [{        "name": "Message",        "type":"record",        "fields": [{            "name": "to",            "type": "string"        },        {            "name": "from",            "type": "string"        },        {            "name": "body",            "type": "string"        }]    }],    "messages": {        "send": {            "request": [{                "name": "message",                "type": "Message"            }],            "response": "string"        }    }}

首先定义了命名空间以及协议名称，然后定义协议用到的数据类型Message，最后定义了一个消息（方法）send。具体的协议定义规范参考Avro 标准中的协议声明。

该文件我们放在src/main/avro/目录中。

创建Maven项目

pom文件如下：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <groupId>me.lin.avro</groupId>    <artifactId>avro-started</artifactId>    <version>0.0.1-SNAPSHOT</version>    <properties>        <avro.version>1.8.1</avro.version>    </properties>    <dependencies>        <dependency>            <groupId>org.apache.avro</groupId>            <artifactId>avro</artifactId>            <version>1.8.1</version>        </dependency>        <dependency>            <groupId>org.apache.avro</groupId>            <artifactId>avro-ipc</artifactId>            <version>${avro.version}</version>        </dependency>    </dependencies>    <build>        <plugins>            <plugin>                <groupId>org.apache.avro</groupId>                <artifactId>avro-maven-plugin</artifactId>                <version>1.8.1</version>                <executions>                    <execution>                        <phase>generate-sources</phase>                        <goals>                            <goal>schema</goal>                            <goal>protocol</goal>                            <goal>idl-protocol</goal>                        </goals>                        <configuration>                            <sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory>                            <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>                        </configuration>                    </execution>                </executions>            </plugin>            <plugin>                <groupId>org.apache.maven.plugins</groupId>                <artifactId>maven-compiler-plugin</artifactId>                <configuration>                    <source>1.7</source>                    <target>1.7</target>                </configuration>            </plugin>        </plugins>        <pluginManagement>            <plugins>                <!--This plugin's configuration is used to store Eclipse m2e settings                    only. It has no influence on the Maven build itself. -->                <plugin>                    <groupId>org.eclipse.m2e</groupId>                    <artifactId>lifecycle-mapping</artifactId>                    <version>1.0.0</version>                    <configuration>                        <lifecycleMappingMetadata>                            <pluginExecutions>                                <pluginExecution>                                    <pluginExecutionFilter>                                        <groupId>                                            org.apache.avro                                        </groupId>                                        <artifactId>                                            avro-maven-plugin                                        </artifactId>                                        <versionRange>                                            [1.8.1,)                                        </versionRange>                                        <goals>                                            <goal>schema</goal>                                            <goal>idl-protocol</goal>                                            <goal>protocol</goal>                                        </goals>                                    </pluginExecutionFilter>                                    <action>                                        <ignore></ignore>                                    </action>                                </pluginExecution>                            </pluginExecutions>                        </lifecycleMappingMetadata>                    </configuration>                </plugin>            </plugins>        </pluginManagement>    </build></project>

主要关注一下插件配置，该配置用于生成代码：

<execution>    <!-- 在代码生成节点运行该插件-->    <phase>generate-sources</phase>    <goals>        <!--运行插件中的schema，protocol，idl-protocol生成代码-->        <goal>schema</goal>        <goal>protocol</goal>        <goal>idl-protocol</goal>    </goals>    <configuration>     <!--配置文件读取位置-->             <sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory><!--代码生成目录-->                           <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>    </configuration></execution>

生成代码

运行Maven的generate-sources，（项目右键–>Run as）,生成如下代码：

这里写图片描述

Message类主要对应接口中定义的数据类型，Mail是协议的接口声明以及相应的回调（客户端使用），我们再avpr文件中声明的协议也被解析成Mail中的一个Protocol对象。

Java实现RPC Server

生成代码之后，我们使用Avro提供的基础设施启动一个RPC服务器，在65111端口上监听。

package me.lin.avro.rpc;import java.io.IOException;import java.net.InetSocketAddress;import me.lin.avro.RpcTest.MailImpl;import org.apache.avro.ipc.NettyServer;import org.apache.avro.ipc.specific.SpecificResponder;public class MailRpcServer {    private static void startServer() throws IOException {        new NettyServer(new SpecificResponder(Mail.class,                new MailImpl()), new InetSocketAddress(65111));    }    public static void main(String[] args) throws IOException {        startServer();        System.out.println ( "server started");    }}

Python客户端实现

首先安装avro库及snappy：

pip install pipsudo apt-get install libsnappy-devpip install python-snappy

客户端发送RPC请求：

import sysimport httplibimport avro.ipc as ipcimport avro.protocol as protocolPROTOCOL = protocol.parse(open("../src/main/resources/avro/mail.avpr").read())server_addr = ('localhost', 65111)class UsageError(Exception):    def __init__(self, value):        self.value = value    def __str__(self):        return repr(self.value)if __name__ == '__main__':    if len(sys.argv) != 4:        raise UsageError("Usage: <to> <from> <body>")    # client code - attach to the server and send a message    client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])    requestor = ipc.Requestor(PROTOCOL, client)    # fill in the Message record and send it    message = dict()    message['to'] = sys.argv[1]    message['from'] = sys.argv[2]    message['body'] = sys.argv[3]    params = dict()    params['message'] = message    print("Result: " + requestor.request('send', params))    # cleanup    client.close()

调用这个客户端发送Mail：

python send_mail.py touser fromuser Hello_Avro_RPC

0 0