ScalaException之No TypeTag available for ***
来源:互联网 发布:骑行组队软件 编辑:程序博客网 时间:2024/05/17 03:36
今天使用Scala编写Spark程序的时候因为需要使用SparkSql建表所以有了如下的代码(代码只是例子,不是现实工程):
import org.apache.spark.{SparkContext, SparkConf}/** * Created by Utopia on 2016/7/13. */object testforjob { def main (args: Array[String]) { val conf = new SparkConf() conf.setAppName("Test") conf.setMaster("local") val sc = new SparkContext(conf) val lines = sc.textFile("C://Users/802/Desktop/uip.txt").map(_.split(",")) val sqlContext = new org.apache.spark.sql.SQLContext(sc) import sqlContext.implicits._ case class UserItemPref(UserID: Int, ItemID: Int, Pref: Int) val useritempref = lines.map(k => UserItemPref(k(0).toInt, k(1).toInt, k(2).trim.toInt)).toDF() useritempref.registerTempTable("UIF") val userid = sqlContext.sql("SELECT UserID FROM UIF") userid.map(t => "Name: " + t).collect(````````````.foreach(println) }}
写完以后运行,出现了下图的错误:
发现不是spark出现问题了 ,而是scala编写出现问题了,在starckoverflow找到了解决方法。就是把case类放在主类外面就行了。
代码:
import org.apache.spark.{SparkContext, SparkConf}/** * Created by Utopia on 2016/7/13. */object testforjob { case class UserItemPref(UserID: Int, ItemID: Int, Pref: Int) def main (args: Array[String]) { val conf = new SparkConf() conf.setAppName("Test") conf.setMaster("local") val sc = new SparkContext(conf) val lines = sc.textFile("C://Users/802/Desktop/uip.txt").map(_.split(",")) val sqlContext = new org.apache.spark.sql.SQLContext(sc) import sqlContext.implicits._ val useritempref = lines.map(k => UserItemPref(k(0).toInt, k(1).toInt, k(2).trim.toInt)).toDF() useritempref.registerTempTable("UIF") val userid = sqlContext.sql("SELECT UserID FROM UIF") userid.map(t => "Name: " + t).collect().foreach(println) }}
得到结果:
0 0
- ScalaException之No TypeTag available for ***
- No TypeTag available for person
- SparkSQL: no typetag available for xxxx
- UDF No TypeTag available for String
- Spark组件之SparkSQL学习1之问题报错No TypeTag available for Person
- SparkSQL: no typetag available for xxxx问题的解决办法
- SparkSQL: no typetag available for xxxx问题的解决办法
- There are no packages available for installation
- There are no packages available for installation
- Dubbo:No provider available for the service
- sublime Text3安装Package Control出错之解决办法:There are no packages available for installation
- No entry for BW_USER in table RSADMIN available
- there is no source code available for the current location
- There is No Source Code Available for the Current Location
- There is no source code available for current location
- no project are available for deployment to this server
- eclipse调试错误No source available for "_dl_debug_state() 解决方案
- Myeclipse:No projects are available for deployment to this server!
- XStream 常用注释 (去子节点集合)
- 圆点切换 14款不同效果的圆点切换特效
- Java接口和继承
- Oracle Database 12c SQL Fundamentals I Exam Guide Exam 1Z0-061 学习笔记
- myeclipse加载mysql问题
- ScalaException之No TypeTag available for ***
- Android Studio2.1.1 Mac版显示行数
- 冒泡排序
- 【flash-01】IE下flash总是置于最前
- HDU 3068 最长回文 最长回文子串
- Android开发遇到的
- 一道值得收藏的java面试题
- caffe BasePrefetchingDataLayer 学习
- UVA1329 合作网络 解题报告