hadoop配置文件加载机制

来源：互联网发布：万国数据是外企吗编辑：程序博客网时间：2024/06/05 05:57

hadoop配置文件加载机制

@(博客文章)[hadoop]

hadoop通过Configuration类来保存配置信息
1、通过Configuration.addResource()来加载配置文件
2、通过Configuration.get***()来获取配置属性

1、创建一个新的Configuration实例时，会加载core-default.xml与core-site.xml，若还有其它配置，则调用addResource()继续添加。

package org.jediael.hadoopdemo;import org.apache.hadoop.conf.Configuration;public class ConfigurationDemo {    public static void main(String[] args){        Configuration conf = new Configuration();        //conf.addResource("/opt/jediael/hadoop-1.2.1/conf/core-site.xml");        Configuration.addDefaultResource("mapred-default.xml");        //System.out.println(conf.get("fs.default.name"));        System.out.println(conf.get("hadoop.tmp.dir"));        System.out.println(conf.get("io.sort.mb"));    }}

输出结果为：

www:Downloads liaoliuqing$ hadoop jar hdemo.jar org.jediael.hadoopdemo.ConfigurationDemohdfs://localhost:9000/opt/tmpnull

2、可以通过Configuration.addDefaultResource（）加载CLASSPATH下的内容，主要用于加载hdfs-default.xml hdfs-site.xml mapred-default.xm mapred-site.xml

package org.jediael.hadoopdemo;import org.apache.hadoop.conf.Configuration;public class ConfigurationDemo {    public static void main(String[] args){        Configuration conf = new Configuration();        //conf.addResource("/opt/jediael/hadoop-1.2.1/conf/core-site.xml");        Configuration.addDefaultResource("mapred-default.xml");        System.out.println(conf.get("fs.default.name"));        System.out.println(conf.get("hadoop.tmp.dir"));        System.out.println(conf.get("io.sort.mb"));    }}

输出结果为：

www:Downloads liaoliuqing$ hadoop jar hdemo.jar org.jediael.hadoopdemo.ConfigurationDemohdfs://localhost:9000/opt/tmp100

若color.xml位于$HADOOP_HOME/conf

package org.jediael.hadoopdemo;import org.apache.hadoop.conf.Configuration;public class ConfigurationDemo {    public static void main(String[] args){        Configuration conf = new Configuration();        //conf.addResource("/opt/jediael/hadoop-1.2.1/conf/color.xml");        Configuration.addDefaultResource("color.xml");        Configuration.addDefaultResource("mapred-default.xml");        System.out.println(conf.get("fs.default.name"));        System.out.println(conf.get("hadoop.tmp.dir"));        System.out.println(conf.get("io.sort.mb"));        System.out.println(conf.get("color"));            }}

输出结果为：

www:Downloads liaoliuqing$ hadoop jar hdemo.jar org.jediael.hadoopdemo.ConfigurationDemohdfs://localhost:9000/opt/tmp100yellow

若color.xml不位于$HADOOP_HOME/conf，则color的输出值为null。

3、conf.addResource（）与Configuration.addDefaultResource（）的区别

0 0