Total size of serialized results of 20 tasks (1088.8 MB) is bigger than spark.driver.maxResultSize (

来源:互联网 发布:windows程序编程 编辑:程序博客网 时间:2024/06/05 02:13

Total size of serialized results of 20 tasks (1088.8 MB) is bigger thanspark.driver.maxResultSize (1024.0 MB)

pypark 2.0 (spark python)  执行中出现上述错误,解决方案:

SparkContext.setSystemProperty('spark.driver.maxResultSize', '10g')



0 0
原创粉丝点击