Hadoop参数性能调优(一)--总述

根据yahoo架构师Milind Bhandarkar在《Hadoop Application Performance Tuning》中的阐述，

 

Hadoop的性能调优主要分为以下六个方面：

 

• Changing number of Map and Reduce tasks

 

• Decrease Intermediate data size using combiner

 

• Decrease map-side disk spill

 

• Decrease intermediate data size by compressing map output

 

• Decrease Reduce-side disk spill

 

• Increase Slots per node