使用IntelliJ IDEA编译spark-1.3.0源码

来源：互联网发布：中国第一程序员陈光威编辑：程序博客网时间：2024/06/07 11:09

1、下源spark源码spark-1.3.0.tgz，并解压；

2、打开IntelliJ IDEA，在欢迎界面上，选择“ImportProject”。

3、在“Select file ordirectory to import”界面中，找到第1步所解压出来的spark-1.3.0源码的根目录下的pom.xml文件。

4、在“Import projectform Maven”界面，使用默认设置，直接选择“Next”；

5、选择所要激活的profiles，如hadoop-2.2、hbase-provided、hive-0.13.1、hive-thriftserver、yarn、parquet-provided。

6、为项目选择SDK：

7、设置project name，这里采用默认的“spark-parent_2.10”。

8、点上图的“Finish”后，IDEA将导入项目。

9、点右侧的“Mav en Projects”，选择“Spark Project Parent POM（root）”，然后点击工具栏倒数第三个按钮（Toggle'Skip Tests' Mode），这时，Lifecycle下的“test”变为灰色，并加上了删除线。

10、点击“Maven Projects”的工具栏的最后一个按钮“MavenSettings”，进入maven的配置界面。为Maven 的 Runner设置 VM Options，并OK保存。

-Xmx512m-XX:MaxPermSize=256M -XX:ReservedCodeCacheSize=512m

11、回到“maven projects”视图，选择“Lifecycle”中的“package”，按第五个按钮“RunMavne Build”，开始进行编译。

12、编译完成后，信息如下：

outputFile=D:\programFile\Hadoop_spark\spark\spark-1.3.0\external\kafka-assembly\scalastyle-output.xml

Processed 0 file(s)

Found 0 errors

Found 0 warnings

Found 0 infos

Finished in 28 ms

[INFO]------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Spark Project Parent POM........................... SUCCESS [ 27.320 s]

[INFO] Spark Project Networking........................... SUCCESS [ 24.204 s]

[INFO] Spark Project Shuffle StreamingService ............ SUCCESS [ 8.997 s]

[INFO] Spark Project Core................................. SUCCESS [06:23 min]

[INFO] Spark Project Bagel................................ SUCCESS [ 19.100 s]

[INFO] Spark Project GraphX............................... SUCCESS [ 44.958 s]

[INFO] Spark Project Streaming............................ SUCCESS [01:13 min]

[INFO] Spark Project Catalyst............................. SUCCESS [01:27 min]

[INFO] Spark Project SQL.................................. SUCCESS [01:34 min]

[INFO] Spark Project ML Library........................... SUCCESS [07:15 min]

[INFO] Spark Project Tools................................ SUCCESS [ 41.362 s]

[INFO] Spark Project Hive .................................SUCCESS [06:09 min]

[INFO] Spark Project REPL................................. SUCCESS [01:48 min]

[INFO] Spark Project YARN................................. SUCCESS [01:57 min]

[INFO] Spark Project Hive Thrift Server................... SUCCESS [01:18 min]

[INFO] Spark Project Assembly............................. SUCCESS [10:33 min]

[INFO] Spark Project External Twitter..................... SUCCESS [ 59.046 s]

[INFO] Spark Project External Flume Sink.................. SUCCESS [01:01 min]

[INFO] Spark Project External Flume....................... SUCCESS [01:20 min]

[INFO] Spark Project External MQTT........................ SUCCESS [01:03 min]

[INFO] Spark Project External ZeroMQ...................... SUCCESS [01:00 min]

[INFO] Spark Project External Kafka....................... SUCCESS [01:43 min]

[INFO] Spark Project Examples............................. SUCCESS [08:43 min]

[INFO] Spark Project YARN Shuffle Service................. SUCCESS [ 38.740 s]

[INFO] Spark Project External KafkaAssembly .............. SUCCESS [02:27 min]

[INFO]------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO]------------------------------------------------------------------------

[INFO] Total time: 01:01 h

[INFO] Finished at:2015-03-23T00:12:18+08:00

[INFO] Final Memory: 72M/375M

[INFO]------------------------------------------------------------------------

Process finished with exit code 0

0 0