Flink学习笔记 --- 理解DataStream WordCount
来源:互联网 发布:cf手游刷图软件 编辑:程序博客网 时间:2024/06/05 23:02
pom,xml 内容如下:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>zetyun</groupId> <artifactId>FlinkWordCounts</artifactId> <version>1.0-SNAPSHOT</version> <inceptionYear>2008</inceptionYear> <properties> <scala.version>2.11.0</scala.version> </properties> <dependencies> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>${scala.version}</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-core --> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-core</artifactId> <version>1.3.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients_2.11 --> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.3.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-scala_2.11 --> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-scala_2.11</artifactId> <version>1.3.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-scala_2.11 --> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-scala_2.11</artifactId> <version>1.3.0</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-core --> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-core</artifactId> <version>0.9.1-hadoop1</version> </dependency> </dependencies></project>
其中的代码如下:
package zetyunimport org.apache.flink.streaming.api.scala._import org.apache.flink.streaming.api.windowing.time.Time/** * Created by ryan on 17-7-19. */object DataStreamWordCount { def main(args: Array[String]) { val env = StreamExecutionEnvironment.getExecutionEnvironment val text = env.socketTextStream("192.168.1.81", 9999) val counts = text.flatMap { _.toLowerCase.split("\\W+") filter { _.nonEmpty } } // convert into lower and filter empty value .map { (_, 1) } // put every char in text into (char, 1) format .keyBy(0) // use the ( char, 1) first element hash function .timeWindow(Time.seconds(5)) // use the window transformation .sum(1) // sum the same key's value counts.print env.execute("Window Stream WordCount") }}
阅读全文
0 0
- Flink学习笔记 --- 理解DataStream WordCount
- Flink学习笔记 --- DataStream Transformations
- Flink学习笔记 --- 理解DataSet WordCount
- Flink学习:DataStream和InputFormat
- Flink学习笔记 --- scala实现Flink的DataSet Source进行WordCount
- Flink学习笔记 --- 理解ConnectedStream 与 Union
- Flink DataStream API Programming Guide学习&译文(未完待续)
- 《Flink学习笔记一》
- Flink学习笔记 --- Flink中Windows机制
- Flink学习笔记:1、Flink快速入门
- Flink学习笔记:2、Flink介绍
- flink 的datastream的作业提交问题
- Flink实现WordCount
- Flink WordCount实例讲解
- mapreduce WordCount 学习笔记
- Flink学习笔记 --- Flink本地(Loacl模式)安装
- Flink学习笔记:3、Flink分布式模式(Standalone)
- Flink学习笔记 --- Basic Concepts整理笔记
- awk打印单双引号
- linux cp命令参数及用法详解---linux 复制文件命令cp
- AOP无法切入同类调用方法的解决办法
- iOS开发-版本管理常用git命令
- 从MultipartFile获取File式的inputStream
- Flink学习笔记 --- 理解DataStream WordCount
- 命令模式
- UVA 11549 Calculator Conundrum Floyd判圈算法 Brent判圈算法 相关性质及证明
- issue-15 了解闭包里如何使用外围作用域里的变量
- HorizontalScrollView 初始化第一次时使用smoothScrollTo无效的解决办法
- SOAPUI测试步骤(七)---DataSink TestStep
- shell脚本
- 括号匹配Java实现
- Java的回调