[问题记录]hdfs删除文件提示rmr: Failed to move to trash

来源：互联网发布：linux时钟同步设置编辑：程序博客网时间：2024/05/28 23:12

问题/现象

使用hadoop fs -rmr xxx删除hdfs上的文件失败

$ hadoop fs -rmr /app/lbs/nuomi-da-stat/stat_platform_auto/1004/dt=20161118/000000_0rmr: Failed to move to trash: /app/lbs/nuomi-da-stat/stat_platform_auto/1004/dt=20161118/000000_0

原因分析

原因一：集群回收站空间占满
原因二：当前ugi没有权限删除对应的hdfs文件，通过hadoop fs -ls命令可以发现，用户权限是nuomi-abc，用户组权限是nuomi-da-stat，当前运行的ugi是在hadoop-site.xml文件中配置的，如果当前运行的ugi与用户权限或用户组权限不符合，就会因为没有权限执行而报错

$ hadoop fs -ls /app/lbs/nuomi-da-stat/stat_platform_auto/1405/dt=20161111/000000_0Found 1 items-rw-r--r--   3 nuomi-abc nuomi-da-stat         85 2016-11-18 05:09 /app/lbs/nuomi-da-stat/stat_platform_auto/1405/dt=20161111/000000_0

解决方案

原因一：集群回收站空间占满

方法：清空集群回收站

$ hadoop fs -rmr /user/nuomi-da-stat/.Trash/*

原因二：当前`ugi`没有权限删除对应的`hdfs`文件

方法一：更改hdfs文件权限

-chown [-R] [OWNER][:[GROUP]] PATH...        Changes owner and group of a file.        This is similar to shell's chown with a few exceptions.    -R  modifies the files recursively. This is the only option        currently supported.        If only owner or group is specified then only owner or        group is modified.        The owner and group names may only cosists of digits, alphabet,        and any of '-_.@/' i.e. [-_.@/a-zA-Z0-9]. The names are case        sensitive.        WARNING: Avoid using '.' to separate user name and group though        Linux allows it. If user names have dots in them and you are        using local file system, you might see surprising results since        shell command 'chown' is used for local files.

如下命令，作用是递归地将/app/lbs/nuomi-da-stat/stat_platform_auto/下的所有文件，用户权限和组权限都改成nuomi-da-stat，文件多的话执行会很慢

hadoop fs -chown -R nuomi-da-stat:nuomi-da-stat /app/lbs/nuomi-da-stat/stat_platform_auto/

方法二：使用有权限的ugi来操作

$ hadoop fs -D hadoop.job.ugi=nuomi-abc -rmr /app/lbs/nuomi-da-stat/stat_platform_auto/1405/dt=20161111/000000_0Moved to trash: /app/lbs/nuomi-da-stat/stat_platform_auto/1405/dt=20161111/000000_0

1 0