Spark-submit脚本

Spark-submit.sh

 

/opt/spark/bin/spark-submit \

--master yarn \

--deploy-mode cluster \

--num-executors  8  \ 集群上启动的executor数

--executor-memory  1g  \

--executor-cores 5  \ 每个executor运行的核数，即一个executor能并行运行的task任务数，

--driver-memory 2g \ 分配给driver的内存数。

/opt/jars/spark/spark-xx.jar \