403 forbidden(Spider error)
来源:互联网 发布:shipyard 增加node 编辑:程序博客网 时间:2024/05/22 06:16
When I run a spider on Taobao get the error 403.
As the above page image, it receive the response successfully.
But if you open the request url on a new web, it will receive 403 forbidden, I do not know what happened there.
I try imitate the request by using postman but also got the 403.
After that, I found that some params like :authority
、 :scheme
and these things make me find that postman not support http2, so I try make it by python requests.
Then an error occured. I throw the symbol colon, then get the real response.
There are still something make me boring.
Anyway, I make it.
Finally, show you the code:
import requestsurl = "https://detailskip.taobao.com/service/getData/1/p1/item/detail/sib.htm"querystring = {"itemId":"537337861521","sellerId":"1096320518","modules":"dynStock,qrcode,viewer,price,contract,duty,xmpPromotion,delivery,upp,activity,fqg,zjys,amountRestriction,couponActivity,soldQuantity,tradeContract","callback":"onSibRequestSuccess"}headers = { 'x-requested-with': "XMLHttpRequest", 'accept': "*/*", 'accept-encoding': "gzip, deflate, sdch, br", 'accept-language': "zh-CN,zh;q=0.8,en-US;q=0.6,en;q=0.4", 'cookie': "thw=cn; t=91a9f63dff91a1c00e8552a2229bf49f; cookie2=1cb5763df42f2835cb43b824eb1fcd18; v=0; cna=; _tb_token_=04eeb53d3511; uc1=cookie14=UoW%2Bvf0SWNDvrQ%3D%3D; l=AsbGqOOlKA9YtY278R47/14olrdIygrl; isg=Ah4epUtxy28OGB8NkAaMWo4rb7SeFuJZFy2HfMinZ2FY67zFMG_naHjLhUWa", 'referer': "https://item.taobao.com/item.htm?spm=a21fg.7938700.204843.1.sGwL2B&scm=1007.12144.81309.2193_1158&pvid=4fad7005-d521-4b2a-a27f-dc3d3bb3e557&id=532801124551", 'user-agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36", 'authority':"detailskip.taobao.com", 'method':"GET", 'path':"/service/getData/1/p1/item/detail/sib.htm?itemId=532801124551&sellerId=367896056&modules=dynStock,qrcode,viewer,price,contract,duty,xmpPromotion,delivery,upp,activity,fqg,zjys,couponActivity,soldQuantity,tradeContract&callback=onSibRequestSuccess", 'scheme':"https", 'cache-control': "no-cache", 'postman-token': "32d300ff-08db-6069-bde5-9d8e091915d1" }response = requests.request("GET", url, headers=headers, params=querystring)print(response.text)
阅读全文
1 0
- 403 forbidden(Spider error)
- “Error 403--Forbidden”错误
- “Error 403--Forbidden”错误
- “Error 403--Forbidden”错误
- “Error 403--Forbidden”错误
- “Error 403--Forbidden”错误
- Error 403--Forbidden
- Apache 403 Forbidden error
- nginx 403 Forbidden Error
- [WordPress] 403 forbidden error
- RewriteEngine causes 403 Forbidden error
- python3 HTTP Error 403:Forbidden
- urllib.error.HTTPError: HTTP Error 403: Forbidden
- Weblogic 403 Error 403--Forbidden错误
- wget URL is forbidden, 403 error
- urllib2.HTTPError: HTTP Error 403: Forbidden
- Apache 403 Forbidden Error and Solution
- urllib2.HTTPError: HTTP Error 403: Forbidden 错误
- spring boot 自学笔记(五) Rabbitmq集成,延时消息队列实现
- 项目中用LOG代替System.out的写法
- linux中hive安装和部署详解
- 【算法题】数组重复元素
- 【已解决】Android studio中ADB启动失败,导致找不到虚拟机或真机
- 403 forbidden(Spider error)
- jQuery的 on() 方法详解
- 常用的系统函数、自定义函数、程序调试
- 索引
- 使用CORS:跨域两三事
- JavaScript组成
- 数据库——(8)
- JavaScript基础
- invalid order function for sorting