(*:update) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml

来源:互联网 发布:数码网络侦探 攻略 编辑:程序博客网 时间:2024/06/05 05:36

SBT异常信息如下:

[info] Loading project definition from D:\IdeaProjects\spark-practice\project[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)> [info] Defining *:shellPrompt[info] The new value will be used by no settings or tasks.[info] Reapplying settings...[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)[info] Defining */*:sbtStructureOutputFile[info] The new value will be used by no settings or tasks.[info] Reapplying settings...[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)[info] Defining */*:sbtStructureOptions[info] The new value will be used by no settings or tasks.[info] Reapplying settings...[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)[info] Applying State transformations org.jetbrains.sbt.CreateTasks from C:/Users/gyg/.IntelliJIdea2016.3/config/plugins/Scala/launcher/sbt-structure-0.13.jar[info] Reapplying settings...[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)[error] Modules were resolved with conflicting cross-version suffixes in {file:/D:/IdeaProjects/spark-practice/}spark-practice:[error]    org.scala-lang.modules:scala-xml _2.12, _2.11[info] Resolving org.scala-sbt#sbt;0.13.15 ...[info] Resolving org.scala-lang#scala-library;2.10.6 ...[info] Resolving org.scala-sbt#main;0.13.15 ...[info] Resolving org.scala-sbt#actions;0.13.15 ...[info] Resolving org.scala-sbt#classpath;0.13.15 ...[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...[info] Resolving org.scala-sbt#interface;0.13.15 ...[info] Resolving org.scala-sbt#io;0.13.15 ...[info] Resolving org.scala-sbt#control;0.13.15 ...[info] Resolving org.scala-sbt#launcher-interface;1.0.1 ...[info] Resolving org.scala-sbt#completion;0.13.15 ...[info] Resolving org.scala-sbt#collections;0.13.15 ...[info] Resolving jline#jline;2.14.3 ...[info] Resolving org.scala-sbt#api;0.13.15 ...[info] Resolving org.scala-sbt#classfile;0.13.15 ...[info] Resolving org.scala-sbt#logging;0.13.15 ...[info] Resolving org.scala-sbt#process;0.13.15 ...[info] Resolving org.scala-sbt#compiler-integration;0.13.15 ...[info] Resolving org.scala-sbt#incremental-compiler;0.13.15 ...[info] Resolving org.scala-sbt#relation;0.13.15 ...[info] Resolving org.scala-sbt#compile;0.13.15 ...[info] Resolving org.scala-sbt#persist;0.13.15 ...[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.15 ...[info] Resolving org.scala-sbt#ivy;0.13.15 ...[info] Resolving org.scala-sbt#cross;0.13.15 ...[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4 ...[info] Resolving com.jcraft#jsch;0.1.50 ...[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...[info] Resolving org.scala-sbt#run;0.13.15 ...[info] Resolving org.scala-sbt#task-system;0.13.15 ...[info] Resolving org.scala-sbt#tasks;0.13.15 ...[info] Resolving org.scala-sbt#tracking;0.13.15 ...[info] Resolving org.scala-sbt#cache;0.13.15 ...[info] Resolving org.scala-sbt#testing;0.13.15 ...[info] Resolving org.scala-sbt#test-agent;0.13.15 ...[info] Resolving org.scala-sbt#test-interface;1.0 ...[info] Resolving org.scala-sbt#main-settings;0.13.15 ...[info] Resolving org.scala-sbt#apply-macro;0.13.15 ...[info] Resolving org.scala-sbt#command;0.13.15 ...[info] Resolving org.scala-sbt#template-resolver;0.1 ...[info] Resolving org.scala-sbt#logic;0.13.15 ...[info] Resolving org.scala-sbt#compiler-interface;0.13.15 ...[info] Resolving org.scala-sbt#api;0.13.15 ...[info] Resolving org.scala-sbt#compile;0.13.15 ...[info] Resolving org.scala-sbt#persist;0.13.15 ...[info] Resolving org.scala-sbt#process;0.13.15 ...[info] Resolving org.scala-sbt#classpath;0.13.15 ...[info] Resolving org.scala-sbt#relation;0.13.15 ...[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...[info] Resolving org.scala-sbt#test-agent;0.13.15 ...[info] Resolving org.scala-sbt#logging;0.13.15 ...[info] Resolving org.scala-sbt#command;0.13.15 ...[info] Resolving org.scala-sbt#task-system;0.13.15 ...[info] Resolving org.scala-sbt#compiler-integration;0.13.15 ...[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...[info] Resolving org.scala-sbt#actions;0.13.15 ...[info] Resolving org.scala-sbt#classfile;0.13.15 ...[info] Resolving org.scala-sbt#ivy;0.13.15 ...[info] Resolving org.scala-sbt#compiler-interface;0.13.15 ...[info] Resolving org.scala-sbt#cache;0.13.15 ...[info] Resolving org.scala-sbt#testing;0.13.15 ...[info] Resolving org.scala-sbt#launcher-interface;1.0.1 ...[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...[info] Resolving org.scala-sbt#cross;0.13.15 ...[info] Resolving org.scala-sbt#tasks;0.13.15 ...[info] Resolving org.scala-sbt#apply-macro;0.13.15 ...[info] Resolving org.scala-sbt#main;0.13.15 ...[info] Resolving org.scala-sbt#collections;0.13.15 ...[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...[info] Resolving org.scala-sbt#io;0.13.15 ...[info] Resolving org.scala-sbt#interface;0.13.15 ...[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...[info] Resolving org.scala-lang#scala-library;2.10.6 ...[info] Resolving org.scala-sbt#template-resolver;0.1 ...[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...[info] Resolving org.scala-sbt#tracking;0.13.15 ...[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...[info] Resolving org.scala-sbt#completion;0.13.15 ...[info] Resolving org.scala-sbt#control;0.13.15 ...[info] Resolving org.scala-sbt#sbt;0.13.15 ...[info] Resolving org.scala-sbt#run;0.13.15 ...[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4 ...[info] Resolving org.scala-sbt#test-interface;1.0 ...[info] Resolving com.jcraft#jsch;0.1.50 ...[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...[info] Resolving jline#jline;2.14.3 ...[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.15 ...[info] Resolving org.scala-sbt#incremental-compiler;0.13.15 ...[info] Resolving org.scala-sbt#logic;0.13.15 ...[info] Resolving org.scala-sbt#main-settings;0.13.15 ...[trace] Stack trace suppressed: run 'last *:update' for the full output.[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.[error] (*:update) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml[error] (*:ssExtractDependencies) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml[error] Total time: 1 s, completed 2017-5-18 1:02:48Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0

我的sbt配置文件

```name := "spark"version := "1.0"scalaVersion := "2.12.2"resolvers += "Cloudera Repository" at "https://repository.cloudera.com/content/repositories/releases/"resolvers += "Spring Plugins Repository" at "http://repo.spring.io/plugins-release/"resolvers += "Maven Repository" at "https://mvnrepository.com/"libraryDependencies += "org.apache.camel" % "camel-test-spring" % "2.10.1" % "test" intransitive()libraryDependencies += "com.google.guava" % "guava" % "16.0.1"libraryDependencies += "commons-codec" % "commons-codec" % "1.10"libraryDependencies += "commons-logging" % "commons-logging" % "1.2"libraryDependencies += "org.codehaus.jackson" % "jackson-mapper-asl" % "1.9.13"libraryDependencies += "jline" % "jline" % "2.12.1"libraryDependencies += "org.apache.zookeeper" % "zookeeper" % "3.4.5-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-auth" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-annotations" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-yarn-server-nodemanager" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.0-cdh5.7.1"libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.12.2"libraryDependencies += "com.typesafe.scala-logging" % "scala-logging_2.11" % "3.5.0" % "provided"libraryDependencies += "org.mongodb" % "casbah_2.11" % "3.1.1"libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0"libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.1.0"

出现error的地方字面理解是Scala版本冲突,即用了2.11版本的Scala,也用了2.12版本的Scala。但是到写这篇博客为止,去mvn中没有找到2.12版本的spark,所以放弃2.12.x版本的Scala,使用了2.11.8之后刷新sbt工程后,下载jar包正常也没有在报错。。

例如上面的
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
其依赖的是
spark-core_2.11

总结一下:
所有jar包依赖的Scala版本号必须一致,不仅spark,还有mongodb等等。
以2.11结尾的jar(如spark-core_2.11)都依赖2.11.x版本的Scala,所以必须用2.11.x版本的Scala,否则会出现Scala包依赖冲突

修改后好用的sbt文件如下:

name := "spark"version := "1.0"scalaVersion := "2.11.8"resolvers += "Cloudera Repository" at "https://repository.cloudera.com/content/repositories/releases/"resolvers += "Spring Plugins Repository" at "http://repo.spring.io/plugins-release/"resolvers += "Maven Repository" at "https://mvnrepository.com/"//  所有jar包依赖的Scala版本号必须一致,//  即所有以2.11结尾的jar(如spark-core_2.11)都依赖2.11.x版本的Scala,所以必须用2.11.x版本的Scala,否则会出现Scala包依赖冲突libraryDependencies += "org.apache.camel" % "camel-test-spring" % "2.10.1" % "test" intransitive()libraryDependencies += "com.google.guava" % "guava" % "16.0.1"libraryDependencies += "commons-codec" % "commons-codec" % "1.10"libraryDependencies += "commons-logging" % "commons-logging" % "1.2"libraryDependencies += "org.codehaus.jackson" % "jackson-mapper-asl" % "1.9.13"libraryDependencies += "jline" % "jline" % "2.12.1"libraryDependencies += "org.apache.zookeeper" % "zookeeper" % "3.4.5-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-auth" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-annotations" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hadoop" % "hadoop-yarn-server-nodemanager" % "2.6.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.0-cdh5.7.1"libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.0-cdh5.7.1"libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.11.8"libraryDependencies += "com.typesafe.scala-logging" % "scala-logging_2.11" % "3.5.0" % "provided"libraryDependencies += "org.mongodb" % "casbah_2.11" % "3.1.1"libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0"libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.1.0"