基于HtmlUnit获取页面及复选框元素、单击事件操作

常用WebClient的参数设置

1. //1.创建对象
2. WebClient webClient=newWebClient(BrowserVersion.CHROME);
3. //2.设置参数
4. //启动js
5. webClient.getOptions().setJavaScriptEnabled(true);
6. //关闭css渲染
7. webClient.getOptions().setCssEnabled(false);
8. //启动重定向
9. webClient.getOptions().setRedirectEnabled(true);
10. //启动cookie管理
11. webClient.setCookieManager(newCookieManager());
12. //启动ajax代理
13. webClient.setAjaxController(newNicelyResynchronizingAjaxController());
14. //js运行时错误，是否抛出异常
15. webClient.getOptions().setThrowExceptionOnScriptError(false);
16. //3.获取页面
17. HtmlPage page=webClient.getPage(url);
18. //等待js渲染执行 waitime等待时间(ms)
19. webClient.waitForBackgroundJavaScript(waitime);

斗鱼页面的获取示例：

    public void getPage(){        String url = Constants.DOU_YU_URL + XMLTools.getNodeValue(Constants.SETTINGS_PATH,"roomID","");        logger.info("请求地址为："+url);//        String url = "http://www.douyu.com";        try {            HtmlPage page = webClient.getPage(url);            //logger.info(page.asText());            DomNodeList domNodeList = page.getElementsByTagName("class");            page.getPageEncoding();            logger.info("页面中class的个数"+domNodeList.size());            logger.info("页面编码："+page.getPageEncoding());            List<HtmlAnchor> listAnchor=  page.getAnchors();            for(HtmlAnchor anchor:listAnchor){                logger.info("页面中链接："+anchor.getHrefAttribute());            }        } catch (IOException e) {            logger.info(e.getMessage());        }

获取页面中复选框及模拟单击事件部分代码：

WebClient初始化及基本设置、获取页面等参考上面。
    URL mainUrl = new URL("https://vendorcentral.amazon.cn/gp/vendor/members/obieeReports/araBasic/ara-basic?ref_=vc_ven-obiee-ara-basic-home_subNav");            HtmlPage mainHtml = webClient.getPage(mainUrl);            //模拟点击跳转            HtmlSelect select = (HtmlSelect) mainHtml.getElementById("report-selector");//            select.setSelectedAttribute("/st/vendor/members/analytics/basic/productDetail",true);            HtmlOption option = select.getOptionByText("销售和库存 - 商品详情");            HtmlPage page3 = option.click();