关于spark streaming新版本ssc创建不了问题

来源:互联网 发布:java项目有红色感叹号 编辑:程序博客网 时间:2024/06/06 13:01

之前因为工作需要于自己需要学习spark streaming,当然官网上介绍的很详细

然而,在我创建:

val ssc = new StreamingContext(sc, Seconds(1))
遇到了问题一直创建不了

官网上给出的包是

importorg.apache.spark.SparkConfimport org.apache.spark.streaming.{Seconds,StreamingContext}

后来创建上首次、时报出了没有这个方法,那就是包没导入

然后就去查了资料发现正确的应该是这样:

import org.apache.spark.streaming._ import org.apache.spark.streaming.StreamingContext._import org.apache.spark.api.java.function._import org.apache.spark.streaming._import org.apache.spark.streaming.api._
// Create a StreamingContext with a local masterval ssc = new StreamingContext(sc, Seconds(1))// Create a DStream that will connect to serverIP:serverPort, like localhost:9999val lines = ssc.socketTextStream("localhost", 9999)// Split each line into wordsval words = lines.flatMap(_.split(" "))import org.apache.spark.streaming.StreamingContext._// Count each word in each batchval pairs = words.map(word => (word, 1))val wordCounts = pairs.reduceByKey(_ + _)// Print a few of the counts to the consolewordCounts.print()ssc.start()             // Start the computationssc.awaitTermination()  // Wait for the computation to terminate

欢迎指教!

0 0
原创粉丝点击