htmlUnit使用

来源：互联网发布：黑帽seo劫持跳转代码编辑：程序博客网时间：2024/06/05 07:46

htmlUnit介绍:

htmlunit 是一款开源的java 页面分析工具，读取页面后，可以有效的使用htmlunit分析页面上的内容。项目可以模拟浏览器运行，被誉为java浏览器的开源实现。是一个没有界面的浏览器，运行速度迅速。官方网址:

http://htmlunit.sourceforge.net/

htmlUnit示例:

pom依赖:

 <dependency>            <groupId>net.sourceforge.htmlunit</groupId>            <artifactId>htmlunit</artifactId>            <version>2.26</version>        </dependency>

测试代码：

import com.gargoylesoftware.htmlunit.WebClient;import com.gargoylesoftware.htmlunit.html.HtmlInput;import com.gargoylesoftware.htmlunit.html.HtmlPage;import org.junit.Test;import java.io.IOException;public class BaiduDemo {    @Test    public void test() throws IOException {        WebClient webClient = new WebClient();        webClient.getOptions().setJavaScriptEnabled(false);        webClient.getOptions().setCssEnabled(false);        webClient.getOptions().setUseInsecureSSL(false);        //获取页面        String url ="https://www.baidu.com";        HtmlPage page = webClient.getPage(url);        System.out.println("页面文本:"+page.getTitleText());        //获取页面元素        HtmlInput htmlInput = page.getHtmlElementById("kw");        System.out.println(htmlInput.asText());        htmlInput.setValueAttribute("test");        HtmlInput btn = page.getHtmlElementById("su");        HtmlPage page2 = btn.click();        System.out.println("页面2:"+page2.getTitleText());    }    }

获取cookie:

BasicCookieStore cookieStore = new BasicCookieStore();        Set<Cookie> cookies2 =webClient.getCookieManager().getCookies();        for (Cookie cookie : cookies2) {            cookieStore.addCookie(cookie.toHttpClient());        }

通过htmlUnit获取的cookie信息发送https请求:

入参:

cookieStore //通过htmlUnit获取的登陆信息

url参数:要访问的页面参数

protected String sendRequest(BasicCookieStore cookieStore,String url) throws Exception {        if (null == cookieStore) {            return null;        }        //设置cookie        HttpClientContext context = HttpClientContext.create();        context.setCookieStore(cookieStore);        //忽略SSL证书验证        SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {            //信任所有            public boolean isTrusted(X509Certificate[] chain,                                     String authType) throws CertificateException {                return true;            }        }).build();          CloseableHttpClient httpClient = HttpClients.custom().setSSLContext(sslContext).build();        HttpGet httpGet = new HttpGet(url);        //发送请求并获取response        CloseableHttpResponse response = httpClient.execute(httpGet, context);        HttpEntity entity = response.getEntity();        return EntityUtils.toString(entity);    }

htmlUnit应用场景

todo

相关资料:

http://blog.csdn.net/lifj07/article/details/8638098

0 0