HttpClient request payload post请求
来源:互联网 发布:停课铃软件 编辑:程序博客网 时间:2024/06/06 11:35
最近在做一些爬虫相关的工作,遇到了一个网站,在使用谷歌开发者工具监控的时候,发现他的请求是这样的
我们都知道传统的post请求,所提交的是form data 格式的,key-value键值对的形式提交。这个在爬虫post请求中非常常见。
第一次遇到了这种形式提交的,后面去找了很多方法,发现在百度搜的答案都用不了,后面翻墙去了stackoverflow,这边贴上地址
http://stackoverflow.com/questions/22671897/httpclient-httppost-and-payload
所给的答案我也用不了。
于是去认真研究了一下,发现他所提交的post流,不是传统的key-value模型的数据,而是post一段json文字
仔细查看了HttpClient的一些相关的API后,找到了答案
这边贴上我的小Demo,Demo是用来抓取某个城市的公积金数据的,这边隐藏去账号密码
String str = "password="+"1234"+"\n"+"theName="+"XXX先生"+"\n"+"year=0\n"+"eTime="+DateUtil.getDate()+"\n"+"sfz="+"idCardxxxx"+"\n"+"orderby=1\n"+"sTime=2000-01-01\n";String loginUrl = "http://www.lszfgjj.gov.cn/ajax/GuestWebGjj_searchUser,App_Web_10rvbinu.ashx?_method=getSearch&_session=r";HttpClient httpClient = new HttpClient();PostMethod post = new PostMethod(loginUrl);post.setRequestHeader("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");post.setRequestHeader("User-Agent","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36");post.setRequestHeader("Accept-Language","zh-cn,zh;q=0.8");post.setRequestHeader("Accept-Encoding","gzip, deflate, sdch");post.setRequestHeader("Content-Type","application/x-www-form-urlencoded; charset=UTF-8");post.setRequestHeader("Host","www.lszfgjj.gov.cn");post.setRequestHeader("Connection","keep-alive");post.setRequestHeader("Referer","http://www.lszfgjj.gov.cn/GuestWebGjj/default.aspx");RequestEntity entity = new StringRequestEntity(str, "text/html", "utf-8");post.setRequestEntity(entity);httpClient.executeMethod(post);String html = post.getResponseBodyAsString();System.out.println(html);比较关键的两步:
RequestEntity entity = new StringRequestEntity(str, "text/html", "utf-8");post.setRequestEntity(entity);
以下是需要import的包:
import org.apache.commons.httpclient.HttpClient;import org.apache.commons.httpclient.methods.PostMethod;import org.apache.commons.httpclient.methods.RequestEntity;import org.apache.commons.httpclient.methods.StringRequestEntity;
希望能帮到各位。
2 0
- HttpClient request payload post请求
- HttpClient request payload post请求
- HttpClient request payload post请求(传输的数据为json)
- http post payload request类型请求
- angularjs中$http模块POST请求request payload转form data
- angularjs中$http模块POST请求request payload转form data
- angularjs中$http模块POST请求request payload转form data
- 【python 爬虫】python如何以request payload形式发送post请求
- vue axios POST请求中参数以form data和request payload形式的原因
- Postman模拟Request Payload发送请求
- HttpClient发送Request请求
- HttpClient--HttpClient的Post请求
- Httpclient发送POST请求
- httpClient发送post请求
- HttpClient发起post请求
- HttpClient post 请求实例
- HttpClient发起POST请求
- httpclient Post请求
- 使用Javamail发送邮件
- Windows下python的xgboost-0.47安装
- 设计模式——6大设计原则
- 2016/10/07 阅读《Practical OpenCV》 char5
- 网易笔试题--最大奇约数
- HttpClient request payload post请求
- 22222222222222
- 栈溢出漏洞利用小结
- 返回引用的函数
- 读书笔记--一个人的朝圣
- UML类图java代码实现
- 70. Climbing Stairs
- 十一
- [经验]Google Nexus5下刷的Android4.4在AS下找不到设备的解决方法