403 forbidden(Spider error)

来源:互联网 发布:shipyard 增加node 编辑:程序博客网 时间:2024/05/22 06:16

这里写图片描述

When I run a spider on Taobao get the error 403.

As the above page image, it receive the response successfully.
But if you open the request url on a new web, it will receive 403 forbidden, I do not know what happened there.

I try imitate the request by using postman but also got the 403.

After that, I found that some params like :authority:scheme and these things make me find that postman not support http2, so I try make it by python requests.

Then an error occured. I throw the symbol colon, then get the real response.

There are still something make me boring.
Anyway, I make it.

Finally, show you the code:

import requestsurl = "https://detailskip.taobao.com/service/getData/1/p1/item/detail/sib.htm"querystring = {"itemId":"537337861521","sellerId":"1096320518","modules":"dynStock,qrcode,viewer,price,contract,duty,xmpPromotion,delivery,upp,activity,fqg,zjys,amountRestriction,couponActivity,soldQuantity,tradeContract","callback":"onSibRequestSuccess"}headers = {    'x-requested-with': "XMLHttpRequest",    'accept': "*/*",    'accept-encoding': "gzip, deflate, sdch, br",    'accept-language': "zh-CN,zh;q=0.8,en-US;q=0.6,en;q=0.4",    'cookie': "thw=cn; t=91a9f63dff91a1c00e8552a2229bf49f; cookie2=1cb5763df42f2835cb43b824eb1fcd18; v=0; cna=; _tb_token_=04eeb53d3511; uc1=cookie14=UoW%2Bvf0SWNDvrQ%3D%3D; l=AsbGqOOlKA9YtY278R47/14olrdIygrl; isg=Ah4epUtxy28OGB8NkAaMWo4rb7SeFuJZFy2HfMinZ2FY67zFMG_naHjLhUWa",    'referer': "https://item.taobao.com/item.htm?spm=a21fg.7938700.204843.1.sGwL2B&scm=1007.12144.81309.2193_1158&pvid=4fad7005-d521-4b2a-a27f-dc3d3bb3e557&id=532801124551",    'user-agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36",    'authority':"detailskip.taobao.com",    'method':"GET",    'path':"/service/getData/1/p1/item/detail/sib.htm?itemId=532801124551&sellerId=367896056&modules=dynStock,qrcode,viewer,price,contract,duty,xmpPromotion,delivery,upp,activity,fqg,zjys,couponActivity,soldQuantity,tradeContract&callback=onSibRequestSuccess",    'scheme':"https",    'cache-control': "no-cache",    'postman-token': "32d300ff-08db-6069-bde5-9d8e091915d1"    }response = requests.request("GET", url, headers=headers, params=querystring)print(response.text)
原创粉丝点击