目录
1、需求
2、注意事项（6点非常重要）
3、建立Project和建立module
4、项目之前的依赖关系
5、代码展示
6、项目中POM依赖展示
7、运行结果展示
—————————————————————————————————————
1、需求
1、IntelliJ IDEA打开多个项目
2、每个同学开发一个项目，相互之前独立不影响
3、通过一个入口可以调用所有项目类、方法、属性，达到同时开发且检测代码
4、dependency只需要写一份，其余项目不用写，便可全部依赖
2、注意事项（非常重要）
6个坑：
1、<groupId>com.yh.bi</groupId>
项目中所有的groupId要一样
2、避免循环依赖，导致程序报错
3、<scope>provided</scope>
打包的服务器运行时候需要provided，本机调试的时候，需要注释
在一个maven项目中，如果存在编译需要而发布不需要的jar包，可以用scope标签，值设为provided
4、项目、module建好之后需要添加Scala的框架支持
5、在yhProject中，可以统一对所有的module进行清理、编译、打包
6、要运行依赖中的module，则必须要将module中的Jar包，打到maven中，需要使用install
下面，是我将所有module中的Jar打到Maven中的路径：
3、建立Project和建立module
1、只需要建立一个项目，其他项目由module建立，所有module且放在项目中。
2、本文项目为yhproject，其余都为module，分别是：mainEntrance、yhutils、yhapp、yhweb、yhgame
项目建立步鄹：
Module建立步鄹：
项目、所有module、部分module代码展示：
4、项目之前的依赖关系
5、代码展示
mainEntrance
package com.yh.bi.dag
package com.yh.bi.dag/** * Created by yuhui on 2017/2/10. */import java.time.{Duration, LocalDate}import com.yh.bi._import com.yh.bi.{UserAPP, UserGame, UserWEB}import org.slf4j.LoggerFactoryimport scala.collection.immutable.{ListMap, ListSet}/** * Created by yuhui on 2016/8/25. * task --> Node --> DAG --> DAGExecutor */case class Node[T](task: T, parent: T*) { override def toString: String = { s"$task(${parent.mkString(",")})" }}case class DAG[T](nodes: Node[T]*)case class DAGExecutor[T](dag: DAG[T]) { private val LOG = LoggerFactory.getLogger(this.getClass) private val _nodes: Map[T, Seq[T]] = dag.nodes.map(node => (node.task, node.parent.filter(_ != null))).toMap private var _pending: Set[T] = ListSet() private var _fails = ListMap[T, String]() private var _success = Seq[T]() private def getPending: Option[T] = { _pending.find { name => val parents = _nodes(name) !parents.exists(name => !_success.contains(name)) } } private def fail(name: T, message: String): Unit = { _pending -= name _fails += name -> message for (child <- _pending.filter(child => _nodes(child).contains(name))) { fail(child, s"依赖的任务无法执行: $name") } } private def success(name: T): Unit = { _pending -= name _success = _success :+ name } def execute(func: T => Unit): Unit = { _pending = _nodes.keySet _fails = ListMap() _success = Seq() var running = true while (running) { val taskOpt = getPending if (taskOpt.nonEmpty) { val task = taskOpt.get val startMills = System.currentTimeMillis() LOG.info("start task {}", task) try { println("=============") func(task) println("+++++++++++++") val time = Duration.ofMillis(System.currentTimeMillis() - startMills) LOG.info(s"end task $task time=$time") success(task) } catch { case e: Throwable => fail(task, e.getMessage) LOG.error(e.getMessage, e) LOG.info(s"fail task $task") } } else { running = false } } for (name <- _success) { LOG.info(s"success task: $name") } for (name <- _fails) { LOG.info(s"fail task: ${name._1} - ${name._2}") } }}object DAG { val allSDKDAG = new DAG[Task]( Node(UserAPP), Node(UserGame), Node(UserWEB) ) def main(args: Array[String]): Unit = { DAGExecutor(allSDKDAG).execute { task =>task.executor("appkey": String, LocalDate.now(), LocalDate.now())} }}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
yhutils
com.yh.bi
package com.yh.bi/** * Created by yuhui on 2017/2/10. */import java.time.LocalDateimport org.apache.spark.sql.SQLContextimport org.slf4j.LoggerFactoryabstract class Executor extends Task with SQLContextAware { override def run(appkey: String, startDay: LocalDate, endDay: LocalDate)={}}trait SQLContextAware { implicit var ctx: SQLContext = _}abstract class Task { protected val LOG = LoggerFactory.getLogger(this.getClass) def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit def run(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = { executor(appkey, startDay, endDay) }}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
yhapp
com.yh.bi
package com.yh.bi/** * Created by yuhui on 2017/2/10. */import java.time.LocalDateobject UserAPP extends Executor{ override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = { println("++++我的UserAPP的执行过程++++") }}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
yhweb
com.yh.bi
package com.yh.biimport java.time.LocalDateobject UserWEB extends Executor{ override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = { println("++++我的UserWEB的执行过程++++") }}
yhgame
com.yh.bi
package com.yh.bi/** * Created by yuhui on 2017/2/10. */import java.time.LocalDateobject UserGame extends Executor{ override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = { println("++++我的UserGame的执行过程++++") }}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
6、项目中POM依赖展示
yhproject中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.yh.bi</groupId> <artifactId>yhproject</artifactId> <packaging>pom</packaging> <version>1.0</version> <modules> <module>mainEntrance</module> <module>yhapp</module> <module>yhweb</module> <module>yhgame</module> <module>yhutils</module> </modules></project>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
mainEntrance中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <artifactId>mainEntrance</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> <dependencies> <dependency> <artifactId>yhutils</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> </dependency> <dependency> <artifactId>yhapp</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> </dependency> <dependency> <artifactId>yhgame</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> </dependency> <dependency> <artifactId>yhweb</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4</version> <configuration> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>**/log4j2.*</exclude> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <outputFile>${project.build.directory}/${project.artifactId}.jar </outputFile> </configuration> </execution> </executions> </plugin> </plugins> <sourceDirectory>src/main/scala</sourceDirectory> <resources> <resource> <directory>${basedir}/src/main/resources</directory> <includes> <include>**/*</include> </includes> </resource> </resources> </build></project>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
yhutils中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>yhproject</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>yhutils</artifactId> <dependencies> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-hive_2.11</artifactId> <version>1.6.1</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4</version> <configuration> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>**/log4j2.*</exclude> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <outputFile>${project.build.directory}/${project.artifactId}.jar </outputFile> </configuration> </execution> </executions> </plugin> </plugins> <sourceDirectory>src/main/scala</sourceDirectory> <resources> <resource> <directory>${basedir}/src/main/resources</directory> <includes> <include>**/*</include> </includes> </resource> </resources> </build></project>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
yhapp中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.yh.bi</groupId> <artifactId>yhapp</artifactId> <version>1.0</version> <dependencies> <dependency> <artifactId>yhutils</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> <scope>provided</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4</version> <configuration> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>**/log4j2.*</exclude> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <outputFile>${project.build.directory}/${project.artifactId}.jar </outputFile> </configuration> </execution> </executions> </plugin> </plugins> <sourceDirectory>src/main/scala</sourceDirectory> <resources> <resource> <directory>${basedir}/src/main/resources</directory> <includes> <include>**/*</include> </includes> </resource> </resources> </build></project>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
yhweb中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.yh.bi</groupId> <artifactId>yhweb</artifactId> <version>1.0</version> <dependencies> <dependency> <artifactId>yhutils</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> <scope>provided</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4</version> <configuration> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>**/log4j2.*</exclude> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <outputFile>${project.build.directory}/${project.artifactId}.jar </outputFile> </configuration> </execution> </executions> </plugin> </plugins> <sourceDirectory>src/main/scala</sourceDirectory> <resources> <resource> <directory>${basedir}/src/main/resources</directory> <includes> <include>**/*</include> </includes> </resource> </resources> </build></project>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
yhgame中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.yh.bi</groupId> <artifactId>yhgame</artifactId> <version>1.0</version> <dependencies> <dependency> <artifactId>yhutils</artifactId> <groupId>com.yh.bi</groupId> <version>1.0</version> <scope>provided</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4</version> <configuration> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>**/log4j2.*</exclude> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <outputFile>${project.build.directory}/${project.artifactId}.jar </outputFile> </configuration> </execution> </executions> </plugin> </plugins> <sourceDirectory>src/main/scala</sourceDirectory> <resources> <resource> <directory>${basedir}/src/main/resources</directory> <includes> <include>**/*</include> </includes> </resource> </resources> </build></project>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
7、运行结果展示
注意：我在mainEntrance执行DAG中的main文件，可以调用且执行了yhutils、yhapp、yhweb、yhgame中的代码
