java爬取某电影网站数据

来源：互联网发布：mac ntfs 新建文件夹编辑：程序博客网时间：2024/06/08 17:30

使用jsoup获取动态网站的数据
这里写图片描述

部分代码：

/** * 获取分类的所有相对链接地址 和名称，保存到map中，返回数据 * */public class GetMoviesName {    private String url;    HashMap<String, String> hrefandname = new HashMap<String, String>();    public GetMoviesName(String url) {        this.url = url;    }    public HashMap<String, String> getAllKinds() throws IOException{        Document kinds = Jsoup.connect(url)                            .userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.78 Safari/537.36")                            .timeout(10000)                            .get();        Elements elements = kinds.select("#content .types a");        for(Element element : elements){            String kindurl = element.attr("href");            hrefandname.put(kindurl, element.text());        }        return hrefandname;    }}

try {            //获取该类别影片的数量total、可在线观看数量playable_count            document = Jsoup.connect(finalurl).timeout(10000).ignoreContentType(true).execute().body();             // document------{"playable_count":18,"total":32,"unwatched_count":32}可在线观看18部，共32部，未观看32部        } catch (IOException e) {            e.printStackTrace();        }

    private int id;    private String name;    private String types;    private String release_date;    private double score;    private String url;    private String is_playable;

源码下载：https://github.com/YanKuan-IT/Jsuop-DouBanMoviesInfo.git

阅读全文

0 0