spark databricks 读入csv数据报错:Multiple sources found for csv

来源:互联网 发布:蓝科cms下载 编辑:程序博客网 时间:2024/06/06 03:46

解决方法参考地址

spark.read.format("org.apache.spark.sql.execution.datasources.csv.CSVFileFormat").format("csv").option("header", "true") // Use first line of all files as header.option("inferSchema", "false") // Automatically infer data types.option("delimiter", ",").load(csvPath)

报错内容:
这里写图片描述

Multiple sources found for csv (org.apache.spark.sql.execution.datasources.csv.CSVFileFormat, com.databricks.spark.csv.DefaultSource15), please specify the fully qualified class name

解决方法:Spark 2.0后需要加入csv的完全路径:

spark.read.format("org.apache.spark.sql.execution.datasources.csv.CSVFileFormat").option("header","true").schema(schema).load(csvPath)
原创粉丝点击