[WeChall] Training: WWW-Robots (HTTP, Training)
来源:互联网 发布:手机图片点击放大js 编辑:程序博客网 时间:2024/06/07 07:08
question:
Training: WWW-Robots (HTTP, Training)
WWW-Robots
In this little training challenge, you are going to learn about the Robots_exclusion_standard.
The robots.txt file is used by web crawlers to check if they are allowed to crawl and index your website or only parts of it.
Sometimes these files reveal the directory structure instead protecting the content from being crawled.
Enjoy!
The robots.txt file is used by web crawlers to check if they are allowed to crawl and index your website or only parts of it.
Sometimes these files reveal the directory structure instead protecting the content from being crawled.
Enjoy!
solution:
go to the website : www.WeChall.net/robots.txt
and we saw this:
User-agent: *Disallow: /challenge/training/www/robots/T0PS3CR
CP the directory after www.WeChall.net and go. SUCCESS!
0 0
- [WeChall] Training: WWW-Robots (HTTP, Training)
- WeChall Training: WWW-Robots
- WeChall Training: WWW-Robots
- [WeChall] Training: ASCII (Training, Encoding)
- WeChall Training: Get Sourced
- WeChall Training: Stegano I
- WeChall Training: ASCII
- WeChall Training: Get Sourced
- WeChall Training: Stegano I
- WeChall Training: ASCII
- [WeChall] Training: Stegano I
- [Wechall]Training:MySQL
- [WeChall] Training: Crypto - Caesar I (Crypto, Training)
- WeChall Training: Crypto - Caesar I
- WeChall Training: Crypto - Caesar I
- wechall php系列之Training: PHP LFI
- Training
- Wechall刷题(二) Training: MySQL I 总结
- web服务器
- 【C#】C#数据类型和VB的区别
- LeetCode 解题报告 LinkedListCycleII
- Java的内存区域
- Java入门到精通——开篇
- [WeChall] Training: WWW-Robots (HTTP, Training)
- 中文分词与隐马尔科夫模型之一:机械分词
- android学习之Service初学
- Ubuntu Linux的虚拟机安装Windows86X虚拟机安装(Ubuntu ISO64Bit)
- 递归及递归的效率问题
- orale性能调优工具--EXPLAIN PLAN和DBMS_XPLAN
- c# webserice 返回DataTable部分数据被截取 显示不全的问题
- 中文分词与马尔科夫模型之二:隐马尔科夫模型与维特比
- Ubuntu Linux访问小米手机存储卡