Experience on Namenode backup and restore --- checkpoint......
来源:互联网 发布:冰镇西瓜 知乎 编辑:程序博客网 时间:2024/05/01 08:24
Hadoop version: Hadoop 2.2.0.2.0.6.0-0009
Well, We can do this by building Secondary Namenode, Checkpoint node or Backup node.
Example:
Assuming you have a Secondary Namenode.
1. Check secondary namenode checkpoint status:
dfs.namenode.secondary.http-address in %HADOOP_CONF_DIR%/hdfs-site.xml
fs.namenode.checkpoint.dir in %HADOOP_CONF_DIR%/hdfs-site.xml
dfs.namenode.checkpoint.edits.dir in %HADOOP_CONF_DIR%/hdfs-site.xml
dfs.namenode.checkpoint.period in %HADOOP_CONF_DIR%/hdfs-site.xml
2. Backup your real time checkpoint by hand:
On Secondary namenode, Stop Hadoop secondary namenode service.
Run cmd.exe by user hadoop ( or some users have full permission )
Runas /user:hadoop cmd.exeYou must have user hadoop password.
Backup real time checkpoint:
cmd>%hadoop_home%/bin/hadoop secondarynamenode -checkpoint forceStart Hadoop secondary namenode service. and check secondary namenode checkpoint status ( see step 1)
3. Stop Namenode services or reboot Namenode ( if hadoop service set to booting manual ,the services would all stop after reboot )
As for test, I backup my dfs.namenode.name.dir (i.e C:\hdpdata\hdfs\nn) first for my next test ( restore from my namenode dir backup ) .
Delete all files in C:\hdpdata\hdfs\nn ,
Open dfs.namenode.checkpoint.dir (see %HADOOP_CONF_DIR%/hdfs-site.xml ) in secondary namenode (i.e. c:\hdpdata\hdfs\snn )
Copy all secondary checkpoint files( except the lock file) from this folder to your namenode's checkpoint dir (dfs.namenode.checkpoint.dir the same as secondary namenode)
Make sure namenode's checkpoint dir is empty already !
4. Restore from checkpoint dir
Run cmd.exe by user hadoop ( or some users have full permission )
Runas /user:hadoop cmd.exeYou must have user hadoop password.
Use this command to start hadoop service and import checkpoint from checkpoint dir
cmd>%hadoop_home%/bin/hdfs namenode -importcheckpoint
Use ctrl+C to stop service which is completed. and Delete your namenode's checkpoint dir (dfs.namenode.checkpoint.dir the same as secondary namenode)
Start service by this command:
cmd>start_local_hdp_services.cmd
Levae safemode
cmd>%hadoop_home%/bin/hdfs dfsadmin -safemode leave
Balance you HDFS:
cmd>%hadoop_home%/bin/hdfs balancer -threshold 5
5. Confirm your Hadoop service is restored successfully.
Open URL http://namenode:50070/ to check if there are some missing block. If yes. Please kindly check where they are and what they are.
Because restore from secondary namenode isn't a real time restore solution. It may lost the last time what you do in the jobtracker. It doesn't matter. Just delete them.
Tips: If you want to restore a real time backup, please use multiplicate namenode dir mode. see next post... ...
douban.com/note/357658915/
douban.com/note/357658944/
douban.com/note/357658985/
douban.com/note/357659006/
douban.com/note/357659032/
douban.com/note/357659066/
douban.com/note/357659080/
douban.com/note/357659115/
douban.com/note/357659146/
douban.com/note/357659171/
douban.com/note/357659262/
douban.com/note/357659308/
douban.com/note/357659332/
douban.com/note/357659368/
douban.com/note/357659402/
douban.com/note/357659438/
douban.com/note/357659452/
douban.com/note/357659498/
douban.com/note/357659534/
douban.com/note/357659551/
douban.com/note/357659728/
douban.com/note/357659772/
douban.com/note/357659796/
douban.com/note/357659822/
douban.com/note/357659845/
douban.com/note/357658699/
douban.com/note/357658751/
douban.com/note/357658812/
douban.com/note/357658864/
douban.com/note/357658884/
douban.com/note/357658939/
douban.com/note/357658974/
douban.com/note/357658999/
douban.com/note/357659026/
douban.com/note/357659048/
douban.com/note/357659070/
douban.com/note/357659096/
douban.com/note/357659103/
douban.com/note/357659194/
douban.com/note/357659215/
douban.com/note/357659243/
douban.com/note/357659272/
douban.com/note/357659321/
douban.com/note/357659351/
douban.com/note/357659381/
douban.com/note/357659421/
douban.com/note/357659443/
douban.com/note/357659489/
douban.com/note/357659625/
douban.com/note/357659642/
douban.com/note/357659692/
douban.com/note/357659753/
douban.com/note/357659800/
douban.com/note/357659836/
- Experience on Namenode backup and restore --- checkpoint......
- Experience on Namenode backup and restore --- checkpoint
- XCloner - Backup and Restore
- Ubuntu backup and restore
- DB2 Backup and Restore
- Sybase backup and restore
- MySQL Backup and Restore
- SQL Server backup and restore
- win 7 backup and restore
- MySQL Database backup and restore
- TFS Express backup and restore
- Backup/Restore Redis and ElasticSearch
- 浅析Hadoop Secondary NameNode,CheckPoint Node,Backup Node
- 浅析Hadoop Secondary NameNode,CheckPoint Node,Backup Node
- 浅析Hadoop Secondary NameNode,CheckPoint Node,Backup Node
- Secondary NameNode,Checkpoint Node,Backup Node,HDFS HA
- HDFS使用Backup Node、Checkpoint Node以及Standby Namenode进行checkpoint的机制详解
- restore hadoop primary namenode from secondary namenode checkpoint step by step
- Hadoop 新 MapReduce 框架 Yarn 详解
- js 添加到收藏夹
- 新人第一次来C博客
- 排序算法实现
- ADF 创建主表子表的先后顺序
- Experience on Namenode backup and restore --- checkpoint......
- 配置Oracle RAC时修改/etc/hosts文件的注意事项
- MyEclipse从数据库反向生成实体类之Hibernate方式 反向工程
- asp.net mvc htmlhelper 扩展方法
- java分页
- 关于Hadoop的shuffle
- JAVA正则表达式语法大全
- tomcat开机自启配置
- 文本框默认值 单击后自动消失