spark 运行模式的简单总结

来源：互联网发布：尿布台有必要买吗知乎编辑：程序博客网时间：2024/06/05 06:48

spark-submit --master spark://192.168.8.19:7077 --deploy-mode client /Users/haozhugogo/Downloads/hdfs_wordcount.py /Users/haozhugogo/Downloads/sparktest.txt//spark自带集群跑 本机是driver，可以看到日志

spark-submit --master spark://192.168.8.19:7077 --deploy-mode cluster /Users/haozhugogo/Downloads/hdfs_wordcount.py /Users/haozhugogo/Downloads/sparktest.txt//spark自带集群跑 其他机器是driver，本机看不到具体日志了

spark-submit --master yarn --deploy-mode client/Users/haozhugogo/Downloads/hdfs_wordcount.py /Users/haozhugogo/Downloads/sparktest.txt //利用yarn集群来跑 本机是driver，可以看到日志

spark-submit --classScalaTest --master yarn --deploy-mode cluster/Users/haozhugogo/Downloads/sparkStreamingTest.jar /Users/haozhugogo/Downloads/sparktest.txt //利用yarn集群来跑 其他机器是driver，本机看不到具体日志了

阅读全文

0 0