Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment variable is set. The Flink YARN Client

来源:互联网 发布:广东人事考试中心 网络 编辑:程序博客网 时间:2024/06/03 16:48

Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment variable is set. The Flink YARN Client needs one of these to be set to properly load the Hadoop configuration for accessing YARN.

Please note that the Client requires the YARN_CONF_DIR or HADOOP_CONF_DIR environment variable to be set to read the YARN and HDFS configuration.

广播变量:
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn/

./bin/yarn-session.sh -n 2 -tm 2000


2017-05-24 18:39:47,775 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: fs.hdfs.hadoopconf, /etc/hadoop/conf.cloudera.yarn
2017-05-24 18:39:47,845 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Using values:
2017-05-24 18:39:47,845 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   -   TaskManager count = 2
2017-05-24 18:39:47,846 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   -   JobManager memory = 1024
2017-05-24 18:39:47,846 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   -   TaskManager memory = 2000
2017-05-24 18:39:49,054 WARN  org.apache.hadoop.util.NativeCodeLoader                       - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-05-24 18:39:49,150 WARN  org.apache.flink.yarn.YarnClusterDescriptor                   - The configuration directory ('/var/lib/hadoop-hdfs/flink-1.2.1/conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.
2017-05-24 18:39:49,170 INFO  org.apache.flink.yarn.Utils                                   - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/conf/logback.xml to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/logback.xml
2017-05-24 18:39:49,626 INFO  org.apache.flink.yarn.Utils                                   - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/conf/log4j.properties to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/log4j.properties
2017-05-24 18:39:49,671 INFO  org.apache.flink.yarn.Utils                                   - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/lib to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/lib
2017-05-24 18:39:51,389 INFO  org.apache.flink.yarn.Utils                                   - Copying from file:/var/lib/hadoop-hdfs/flink-1.2.1/lib/flink-dist_2.10-1.2.1.jar to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/flink-dist_2.10-1.2.1.jar
2017-05-24 18:39:53,143 INFO  org.apache.flink.yarn.Utils                                   - Copying from /var/lib/hadoop-hdfs/flink-1.2.1/conf/flink-conf.yaml to hdfs://cluster1/user/root/.flink/application_1495186603292_0062/flink-conf.yaml
2017-05-24 18:39:53,203 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Submitting application master application_1495186603292_0062
2017-05-24 18:39:53,280 INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Submitted application application_1495186603292_0062
2017-05-24 18:39:53,280 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Waiting for the cluster to be allocated
2017-05-24 18:39:53,284 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deploying cluster, current state ACCEPTED
  17-05-24 18:40:05,290 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - YARN application has been deployed successfully.

启动程序成功。
阅读全文
0 0
原创粉丝点击