spark analysis practice

来源:互联网 发布:淘宝卖面膜怎么样 编辑:程序博客网 时间:2024/06/01 14:25

Update the file to hdfs directory first.

import org.apache.spark.sql.catalyst.encoders.ExpressionEncoderimport org.apache.spark.sql.Encoder// For implicit conversions from RDDs to DataFramesimport spark.implicits._case class Call(day:String, time: String, src: String)val callDF = sc.textFile("iptables/calltable.txt").map(_.split(" ")).map(attributes => Call(attributes(0).trim, attributes(1).trim, attributes(2).trim)).toDF()// Register the DataFrame as a temporary viewcallDF.createOrReplaceTempView("call")



0 0
原创粉丝点击