One Cause of java.net.SocketTimeoutException: Read timed out

来源:互联网 发布:python import thread 编辑:程序博客网 时间:2024/06/10 20:04

When I try to get document from a website using jsoup, I got the error after seconds of stucking.

java.net.SocketTimeoutException: Read timed outat java.net.SocketInputStream.socketRead0(Native Method)at java.net.SocketInputStream.read(Unknown Source)at java.net.SocketInputStream.read(Unknown Source)at java.io.BufferedInputStream.fill(Unknown Source)at java.io.BufferedInputStream.read1(Unknown Source)at java.io.BufferedInputStream.read(Unknown Source)at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)at java.net.HttpURLConnection.getResponseCode(Unknown Source)at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:453)at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)at org.jsoup.helper.HttpConnection.get(HttpConnection.java:170)at com.yystar.crown.adapter.service.CrownScraper.main(CrownScraper.java:12)

Code is as simple as following:

public static void main(String[] args) {String url = "http://xxx.com/";try {Document doc = Jsoup.connect(url).get();System.out.println(doc.data());} catch (IOException e) {e.printStackTrace();}}

What could cause this error? Given that we can access the page normally using browser.

After comparison with browser's HTTP request header(METHODOLOGY), I set theUSER AGENT and issue the request again, surprised, the server responded with the right document. The following snippet works:

public static void main(String[] args) {String url = "http://xxx.com/xxx";try {Document doc = Jsoup.connect(url).followRedirects(true).referrer("http://xxx.com/xxx").userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36").get();System.out.println(doc.data());} catch (IOException e) {e.printStackTrace();}}
Note that the user agent value copied from browser developer tool.






0 0
原创粉丝点击