spark入门学习(1)---利用akka建立最基础的通信框架
来源:互联网 发布:linux mount ntfs 编辑:程序博客网 时间:2024/05/19 17:51
1、前情导要
首先在学习这个系列之前,scala是必须掌握的基础,由于scala和java的无缝结合以及本身具有的函数编程的思想,所以scala的使用是非常灵活的,但是同时,学好scala也是非常有挑战性的。
2、基础模型图
本次简单的实现,就没有用到zk进行集群部署
3、依赖
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>tianjun.cmcc.org</groupId> <artifactId>akka-rpc</artifactId> <version>2.0</version> <properties> <maven.compiler.source>1.7</maven.compiler.source> <maven.compiler.target>1.7</maven.compiler.target> <encoding>UTF-8</encoding> <scala.version>2.10.6</scala.version> <scala.compat.version>2.10</scala.compat.version> </properties> <dependencies> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>${scala.version}</version> </dependency> <dependency> <groupId>com.typesafe.akka</groupId> <artifactId>akka-actor_2.10</artifactId> <version>2.3.14</version> </dependency> <dependency> <groupId>com.typesafe.akka</groupId> <artifactId>akka-remote_2.10</artifactId> <version>2.3.14</version> </dependency> </dependencies> <build> <sourceDirectory>src/main/scala</sourceDirectory> <testSourceDirectory>src/test/scala</testSourceDirectory> <plugins> <plugin> <groupId>net.alchim31.maven</groupId> <artifactId>scala-maven-plugin</artifactId> <version>3.2.2</version> <executions> <execution> <goals> <goal>compile</goal> <goal>testCompile</goal> </goals> <configuration> <args> <arg>-make:transitive</arg> <arg>-dependencyfile</arg> <arg>${project.build.directory}/.scala_dependencies</arg> </args> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4.3</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer"> <resource>reference.conf</resource> </transformer> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>org.tianjun.rpc.Worker</mainClass> </transformer> </transformers> </configuration> </execution> </executions> </plugin> </plugins> </build></project>
4、master部分
package org.tianjun.rpcimport akka.actor.{Props, ActorSystem, Actor}import com.typesafe.config.ConfigFactory/** * 基于akka的rpc * Created by tianjun on 2017/1/11 0011. */class Master extends Actor{ println("constructor invoked!") override def preStart(): Unit = { println("preStart invoked!") } override def receive: Receive = { case "connect" => { println("a client connected") sender ! "reply" } case "hello" => { println("hello") } }}object Master{ def main(args: Array[String]) { val host = args(0) val port = args(1).toInt val configStr = s""" |akka.actor.provider = "akka.remote.RemoteActorRefProvider" |akka.remote.netty.tcp.hostname = "$host" |akka.remote.netty.tcp.port = "$port" """.stripMargin val config = ConfigFactory.parseString(configStr) //ActorSystem老大,辅助创建和监控actor,它是单例的 val actorSystem = ActorSystem("MasterSystem",config) val master = actorSystem.actorOf(Props(new Master),"Master") //master自己给自己发信息 master ! "hello" actorSystem.awaitTermination() }}
5、worker部分
package org.tianjun.rpcimport akka.actor.{Props, ActorSystem, ActorSelection, Actor}import com.typesafe.config.ConfigFactory/** * rpc入门 * Created by tianjun on 2017/1/11 0011. */class Worker(val masterHost:String,val masterPort:Int) extends Actor{ var master:ActorSelection = _ override def preStart(): Unit = { master = context.actorSelection(s"akka.tcp://MasterSystem@$masterHost:$masterPort/user/Master") master ! "connect" } override def receive: Receive = { case "reply" => { println("reply from master") } }}object Worker{ def main(args: Array[String]) { val host = args(0) val port = args(1).toInt val masterHost=args(2) val masterPort = args(3).toInt val configStr = s""" |akka.actor.provider = "akka.remote.RemoteActorRefProvider" |akka.remote.netty.tcp.hostname = "$host" |akka.remote.netty.tcp.port = "$port" """.stripMargin val config = ConfigFactory.parseString(configStr) //ActorSystem老大,辅助创建和监控actor,它是单例的 val actorSystem = ActorSystem("WorkSystem",config) actorSystem.actorOf(Props(new Worker(masterHost,masterPort)),"Worker") actorSystem.awaitTermination() }}
6、单机运行:
master
[INFO] [01/11/2017 15:33:22.941] [main] [Remoting] Starting remoting[INFO] [01/11/2017 15:33:23.167] [main] [Remoting] Remoting started; listening on addresses :[akka.tcp://MasterSystem@192.168.64.1:8888][INFO] [01/11/2017 15:33:23.168] [main] [Remoting] Remoting now listens on addresses: [akka.tcp://MasterSystem@192.168.64.1:8888]constructor invoked!preStart invoked!helloa client connected
worker
[INFO] [01/11/2017 15:33:26.312] [main] [Remoting] Starting remoting[INFO] [01/11/2017 15:33:26.465] [main] [Remoting] Remoting started; listening on addresses :[akka.tcp://WorkSystem@192.168.64.1:9999][INFO] [01/11/2017 15:33:26.466] [main] [Remoting] Remoting now listens on addresses: [akka.tcp://WorkSystem@192.168.64.1:9999]reply from master
笔者通过改变pom 文件中的MainClass,把master和woker分别丢到两台不通的服务器,也是ok的,这里就可以知道,其实基于akka智商的通信是比较简单的,大家可以比对前面基于netty的rpc就可以知道(akka底层通信也是通过netty来实现的)
0 0
- spark入门学习(1)---利用akka建立最基础的通信框架
- spark入门学习(2)---利用akka建立基于心跳基础的通信框架
- Spark内部通信/akka
- Spark为何使用Netty通信框架替代Akka
- 通信框架AKKA介绍
- Spark:Akka入门初窥
- Spark入门到精通视频学习资料--第四章:基于Spark的流处理框架Spark Streaming(2讲)
- akka基础学习
- Spark:Akka实现简单RPC通信
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- PowerDesigner最基础的使用方法入门学习
- JavaScript之数组去重
- 白话Https
- AndroidMainfest.xml详解——<receiver>
- Spring MVC
- MS DataGrid导出Excel
- spark入门学习(1)---利用akka建立最基础的通信框架
- Proc *C/C++入门之指示器变量
- query.scroll()和query.setFirstResult(),query.setMaxResults();
- 17 - 03 - 14 图解HTTP(18)
- LSGO代码小组第18周复盘日志
- 研究生周报(第五次) 2016/11/05-2016/11/11
- 《程序员的修炼之道》读书笔记
- 第三方支付为何不能实时出款
- Android如何开发自定义编译时注解