本地运行hadoop-Failed to locate the winutils binary in the hadoop binary path-转载
来源:互联网 发布:我的世界工业js包 编辑:程序博客网 时间:2024/06/06 20:37
转自:http://www.cnblogs.com/zq-inlook/p/4386216.html
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的。因为工作需要,需要在windows上先调试该程序,然后再转到linux下。程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable
null
\bin\winutils.exe in the Hadoop binaries.
通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。
privatestatic String checkHadoopHome() { // first check the Dflag hadoop.home.dir with JVM scope String home = System.getProperty("hadoop.home.dir"); // fall back to the system/user-global env variableif (home == null) { home = System.getenv("HADOOP_HOME"); } try { // couldn't find either setting for hadoop's home directoryif (home == null) { thrownew IOException("HADOOP_HOME or hadoop.home.dir are not set."); } if (home.startsWith("\"") && home.endsWith("\"")) { home = home.substring(1, home.length()-1); } // check that the home setting is actually a directory that exists File homedir = new File(home); if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) { thrownew IOException("Hadoop home directory " + homedir + " does not exist, is not a directory, or is not an absolute path."); } home = homedir.getCanonicalPath(); } catch (IOException ioe) { if (LOG.isDebugEnabled()) { LOG.debug("Failed to detect a valid hadoop home directory", ioe); } home = null; } return home; }
private static String HADOOP_HOME_DIR = checkHadoopHome();
publicstaticfinal String getQualifiedBinPath(String executable) throws IOException { // construct hadoop bin path to the specified executable String fullExeName = HADOOP_HOME_DIR + File.separator + "bin" + File.separator + executable; File exeFile = new File(fullExeName); if (!exeFile.exists()) { thrownew IOException("Could not locate executable " + fullExeName + " in the Hadoop binaries."); } return exeFile.getCanonicalPath(); } /** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath(); publicstaticfinal String getWinUtilsPath() { String winUtilsPath = null; try { if (WINDOWS) { winUtilsPath = getQualifiedBinPath("winutils.exe"); } } catch (IOException ioe) { LOG.error("Failed to locate the winutils binary in the hadoop binary path", ioe); } return winUtilsPath; }
找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:
1.下载winutils的windows版本
GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录
2.配置环境变量
增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。
再次运行程序,正常执行。
- 本地运行hadoop-Failed to locate the winutils binary in the hadoop binary path-转载
- 本地运行hadoop-Failed to locate the winutils binary in the hadoop binary path
- Hadoop报错:Failed to locate the winutils binary in the hadoop binary path
- WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
- WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
- WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
- win7系统下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
- Failed to locate the winutils binary in the hadoop binary path
- spark: ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
- 解决spark运行中ERROR Shell:Failed to locate the winutils binary in the hadoop binary path的问题
- Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locat
- IDEA开发Spark提示failed to locate the winutils binary in the hadoop binary
- 出现 ERROR org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary pa
- windows中以本地模式运行spark遇到“Could not locate executable null\bin\winutils.exe in the Hadoop binarie”
- 解Windows系统下运行hadoop、spark程序出错Could not locate executablenull\bin\winutils.exe in the Hadoop binaries
- spark运行过程中报错Could not locate executable null\bin\winutils.exe in the Hadoop binaries.解决
- Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
- Could not locate executable null\bin\winutils.exe in the Hadoop binaries
- android 自定义控件初步
- iOS UICollectionView 高级用法(长按cell移动重新排列)
- 找最接近小数的分数
- Java的强引用、软引用、弱引用、虚引用
- Android 4.4 Graphic系统详解(1) SurfaceFlinger的启动过程
- 本地运行hadoop-Failed to locate the winutils binary in the hadoop binary path-转载
- mybatis执行批量更新batch update 的方法(oracle,mysql)
- 在xib里,拖一个UIView到UITableView中作为tableFooterView
- Apache jar包总结
- 二十岁出头的时候上,你一无所有,你拥有一切。
- Java笔记
- R语言之数据可视化
- Android 4.4 Graphic系统详解(2) VSYNC的生成
- Map.containsKey方法——判断Map集合对象中是否包含指定的键名