OkHttp3中的代理与路由
来源:互联网 发布:在淘宝开店怎么收费 编辑:程序博客网 时间:2024/05/19 02:40
HTTP请求的整体处理过程大体可以理解为,
- 建立TCP连接。
- 如果是HTTPS的话,完成SSL/TLS的协商。
- 发送请求。
- 获取响应。
- 结束请求,关闭连接。
然而,当为系统设置了代理的时候,整个数据流都会经过代理服务器。那么代理设置究竟是如何工作的呢?它是如何影响我们上面看到的HTTP请求的处理过程的呢?是在操作系统内核的TCP实现中的策略呢,还是HTTP stack中的机制?这里我们通过OkHttp3中的实现来一探究竟。
代理分为两种类型,一种是SOCKS代理,另一种是HTTP代理。对于SOCKS代理,在HTTP的场景下,代理服务器完成TCP数据包的转发工作。而HTTP代理服务器,在转发数据之外,还会解析HTTP的请求及响应,并根据请求及响应的内容做一些处理。这里看一下OkHttp中对代理的处理。
代理服务器的描述
在Java中,通过 java.net.Proxy 类描述一个代理服务器:
public class Proxy { /** * Represents the proxy type. * * @since 1.5 */ public enum Type { /** * Represents a direct connection, or the absence of a proxy. */ DIRECT, /** * Represents proxy for high level protocols such as HTTP or FTP. */ HTTP, /** * Represents a SOCKS (V4 or V5) proxy. */ SOCKS }; private Type type; private SocketAddress sa; /** * A proxy setting that represents a {@code DIRECT} connection, * basically telling the protocol handler not to use any proxying. * Used, for instance, to create sockets bypassing any other global * proxy settings (like SOCKS): * <P> * {@code Socket s = new Socket(Proxy.NO_PROXY);} * */ public final static Proxy NO_PROXY = new Proxy(); // Creates the proxy that represents a {@code DIRECT} connection. private Proxy() { type = Type.DIRECT; sa = null; } /** * Creates an entry representing a PROXY connection. * Certain combinations are illegal. For instance, for types Http, and * Socks, a SocketAddress <b>must</b> be provided. * <P> * Use the {@code Proxy.NO_PROXY} constant * for representing a direct connection. * * @param type the {@code Type} of the proxy * @param sa the {@code SocketAddress} for that proxy * @throws IllegalArgumentException when the type and the address are * incompatible */ public Proxy(Type type, SocketAddress sa) { if ((type == Type.DIRECT) || !(sa instanceof InetSocketAddress)) throw new IllegalArgumentException("type " + type + " is not compatible with address " + sa); this.type = type; this.sa = sa; } /** * Returns the proxy type. * * @return a Type representing the proxy type */ public Type type() { return type; } /** * Returns the socket address of the proxy, or * {@code null} if its a direct connection. * * @return a {@code SocketAddress} representing the socket end * point of the proxy */ public SocketAddress address() { return sa; }......}
只用代理的类型及代理服务器地址即可描述代理服务器的全部。对于HTTP代理,代理服务器地址可以通过域名和IP地址等方式来描述。
代理选择器ProxySelector
在Java中通过ProxySelector为一个特定的URI选择代理:
public abstract class ProxySelector {...... /** * Selects all the applicable proxies based on the protocol to * access the resource with and a destination address to access * the resource at. * The format of the URI is defined as follow: * <UL> * <LI>http URI for http connections</LI> * <LI>https URI for https connections * <LI>{@code socket://host:port}<br> * for tcp client sockets connections</LI> * </UL> * * @param uri * The URI that a connection is required to * * @return a List of Proxies. Each element in the * the List is of type * {@link java.net.Proxy Proxy}; * when no proxy is available, the list will * contain one element of type * {@link java.net.Proxy Proxy} * that represents a direct connection. * @throws IllegalArgumentException if the argument is null */ public abstract List<Proxy> select(URI uri); /** * Called to indicate that a connection could not be established * to a proxy/socks server. An implementation of this method can * temporarily remove the proxies or reorder the sequence of * proxies returned by {@link #select(URI)}, using the address * and the IOException caught when trying to connect. * * @param uri * The URI that the proxy at sa failed to serve. * @param sa * The socket address of the proxy/SOCKS server * * @param ioe * The I/O exception thrown when the connect failed. * @throws IllegalArgumentException if either argument is null */ public abstract void connectFailed(URI uri, SocketAddress sa, IOException ioe);}
这个组件会读区系统中配置的所有代理,并根据调用者传入的URI,返回特定的代理服务器集合。由于不同系统中,配置代理服务器的方法,及相关配置的保存机制不同,该接口在不同的系统中有着不同的实现。
OkHttp3的路由
OkHttp3中抽象出Route来描述网络数据包的传输路径,最主要还是要描述直接与其建立TCP连接的目标端点。
public final class Route { final Address address; final Proxy proxy; final InetSocketAddress inetSocketAddress; public Route(Address address, Proxy proxy, InetSocketAddress inetSocketAddress) { if (address == null) { throw new NullPointerException("address == null"); } if (proxy == null) { throw new NullPointerException("proxy == null"); } if (inetSocketAddress == null) { throw new NullPointerException("inetSocketAddress == null"); } this.address = address; this.proxy = proxy; this.inetSocketAddress = inetSocketAddress; } public Address address() { return address; } /** * Returns the {@link Proxy} of this route. * * <strong>Warning:</strong> This may disagree with {@link Address#proxy} when it is null. When * the address's proxy is null, the proxy selector is used. */ public Proxy proxy() { return proxy; } public InetSocketAddress socketAddress() { return inetSocketAddress; } /** * Returns true if this route tunnels HTTPS through an HTTP proxy. See <a * href="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817, Section 5.2</a>. */ public boolean requiresTunnel() { return address.sslSocketFactory != null && proxy.type() == Proxy.Type.HTTP; }......}
主要通过 代理服务器的信息proxy ,及 连接的目标地址 描述路由。 连接的目标地址inetSocketAddress 根据代理类型的不同而有着不同的含义,这主要是由不同代理协议的差异而造成的。对于无需代理的情况, 连接的目标地址inetSocketAddress 中包含HTTP服务器经过了DNS域名解析的IP地址及协议端口号;对于SOCKS代理,其中包含HTTP服务器的域名及协议端口号;对于HTTP代理,其中则包含代理服务器经过域名解析的IP地址及端口号。
路由选择器RouteSelector
HTTP请求处理过程中所需的TCP连接建立过程,主要是找到一个Route,然后依据代理协议的规则与特定目标建立TCP连接。对于无代理的情况,是与HTTP服务器建立TCP连接;对于SOCKS代理及HTTP代理,是与代理服务器建立TCP连接,虽然都是与代理服务器建立TCP连接,而SOCKS代理协议与HTTP代理协议做这个动作的方式又会有一定的区别。
借助于域名解析做负载均衡已经是网络中非常常见的手法了,因而,常常会有相同域名对应不同IP地址的情况。同时相同系统也可以设置多个代理,这使Route的选择变得复杂起来。
在OkHttp中,对Route连接失败有一定的错误处理机制。OkHttp会逐个尝试找到的Route建立TCP连接,直到找到可用的那一个。这同样要求,对Route信息有良好的管理。
OkHttp3借助于 RouteSelector
类管理所有的路由信息,并帮助选择路由。 RouteSelector
主要完成3件事:
收集所有可用的路由。
public final class RouteSelector {private final Address address;private final RouteDatabase routeDatabase;/* The most recently attempted route. */private Proxy lastProxy;private InetSocketAddress lastInetSocketAddress;/* State for negotiating the next proxy to use. */private List<Proxy> proxies = Collections.emptyList();private int nextProxyIndex;/* State for negotiating the next socket address to use. */private List<InetSocketAddress> inetSocketAddresses = Collections.emptyList();private int nextInetSocketAddressIndex;/* State for negotiating failed routes */private final List<Route> postponedRoutes = new ArrayList<>();public RouteSelector(Address address, RouteDatabase routeDatabase) { this.address = address; this.routeDatabase = routeDatabase; resetNextProxy(address.url(), address.proxy());}....../** Prepares the proxy servers to try. */private void resetNextProxy(HttpUrl url, Proxy proxy) { if (proxy != null) { // If the user specifies a proxy, try that and only that. proxies = Collections.singletonList(proxy); } else { // Try each of the ProxySelector choices until one connection succeeds. If none succeed // then we'll try a direct connection below. proxies = new ArrayList<>(); List<Proxy> selectedProxies = address.proxySelector().select(url.uri()); if (selectedProxies != null) proxies.addAll(selectedProxies); // Finally try a direct connection. We only try it once! proxies.removeAll(Collections.singleton(Proxy.NO_PROXY)); proxies.add(Proxy.NO_PROXY); } nextProxyIndex = 0;}
收集路由分为两个步骤:第一步收集所有的代理;第二步则是收集特定代理服务器选择情况下的所有 连接的目标地址 。
收集代理的过程如上面的这段代码所示,有两种方式,一是外部通过address传入了代理,此时代理集合将包含这唯一的代理。address的代理最终来源于OkHttpClient,我们可以在构造OkHttpClient时设置代理,来指定由该client执行的所有请求经过特定的代理。
另一种方式是,借助于ProxySelector获取多个代理。ProxySelector最终也来源于OkHttpClient,OkHttp的用户当然也可以对此进行配置。但通常情况下,使用系统默认的ProxySelector,来获取系统中配置的代理。
收集到的所有代理保存在列表proxies
中。
为OkHttpClient配置Proxy或ProxySelector的场景大概是,需要让连接使用代理,但不使用系统的代理配置的情况。
收集特定代理服务器选择情况下的所有路由,因代理类型的不同而有着不同的过程:/** Returns the next proxy to try. May be PROXY.NO_PROXY but never null. */private Proxy nextProxy() throws IOException { if (!hasNextProxy()) { throw new SocketException("No route to " + address.url().host() + "; exhausted proxy configurations: " + proxies); } Proxy result = proxies.get(nextProxyIndex++); resetNextInetSocketAddress(result); return result;}/** Prepares the socket addresses to attempt for the current proxy or host. */private void resetNextInetSocketAddress(Proxy proxy) throws IOException { // Clear the addresses. Necessary if getAllByName() below throws! inetSocketAddresses = new ArrayList<>(); String socketHost; int socketPort; if (proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.SOCKS) { socketHost = address.url().host(); socketPort = address.url().port(); } else { SocketAddress proxyAddress = proxy.address(); if (!(proxyAddress instanceof InetSocketAddress)) { throw new IllegalArgumentException( "Proxy.address() is not an " + "InetSocketAddress: " + proxyAddress.getClass()); } InetSocketAddress proxySocketAddress = (InetSocketAddress) proxyAddress; socketHost = getHostString(proxySocketAddress); socketPort = proxySocketAddress.getPort(); } if (socketPort < 1 || socketPort > 65535) { throw new SocketException("No route to " + socketHost + ":" + socketPort + "; port is out of range"); } if (proxy.type() == Proxy.Type.SOCKS) { inetSocketAddresses.add(InetSocketAddress.createUnresolved(socketHost, socketPort)); } else { // Try each address for best behavior in mixed IPv4/IPv6 environments. List<InetAddress> addresses = address.dns().lookup(socketHost); for (int i = 0, size = addresses.size(); i < size; i++) { InetAddress inetAddress = addresses.get(i); inetSocketAddresses.add(new InetSocketAddress(inetAddress, socketPort)); } } nextInetSocketAddressIndex = 0;}/*** Obtain a "host" from an {@link InetSocketAddress}. This returns a string containing either an* actual host name or a numeric IP address.*/// Visible for testingstatic String getHostString(InetSocketAddress socketAddress) { InetAddress address = socketAddress.getAddress(); if (address == null) { // The InetSocketAddress was specified with a string (either a numeric IP or a host name). If // it is a name, all IPs for that name should be tried. If it is an IP address, only that IP // address should be tried. return socketAddress.getHostName(); } // The InetSocketAddress has a specific address: we should only try that address. Therefore we // return the address and ignore any host name that may be available. return address.getHostAddress();}
收集一个特定代理服务器选择下的 连接的目标地址 因代理类型的不同而不同,这主要分为3种情况。 对于没有配置代理的情况,会对HTTP服务器的域名进行DNS域名解析,并为每个解析到的IP地址创建 连接的目标地址;对于SOCKS代理,直接以HTTP服务器的域名及协议端口号创建 连接的目标地址;而对于HTTP代理,则会对HTTP代理服务器的域名进行DNS域名解析,并为每个解析到的IP地址创建 连接的目标地址。
这里是OkHttp中发生DNS域名解析唯一的场合。对于使用代理的场景,没有对HTTP服务器的域名做DNS域名解析,也就意味着HTTP服务器的域名解析要由代理服务器完成。
代理服务器的收集是在创建RouteSelector
完成的;而一个特定代理服务器选择下的 连接的目标地址 收集则是在选择Route时根据需要完成的。RouteSelector
做的第二件事情是选择可用的路由。/*** Returns true if there's another route to attempt. Every address has at least one route.*/public boolean hasNext() {return hasNextInetSocketAddress() || hasNextProxy() || hasNextPostponed();}public Route next() throws IOException {// Compute the next route to attempt.if (!hasNextInetSocketAddress()) { if (!hasNextProxy()) { if (!hasNextPostponed()) { throw new NoSuchElementException(); } return nextPostponed(); } lastProxy = nextProxy();}lastInetSocketAddress = nextInetSocketAddress();Route route = new Route(address, lastProxy, lastInetSocketAddress);if (routeDatabase.shouldPostpone(route)) { postponedRoutes.add(route); // We will only recurse in order to skip previously failed routes. They will be tried last. return next();}return route;}/** Returns true if there's another proxy to try. */private boolean hasNextProxy() {return nextProxyIndex < proxies.size();}/** Returns true if there's another socket address to try. */private boolean hasNextInetSocketAddress() {return nextInetSocketAddressIndex < inetSocketAddresses.size();}/** Returns the next socket address to try. */private InetSocketAddress nextInetSocketAddress() throws IOException {if (!hasNextInetSocketAddress()) { throw new SocketException("No route to " + address.url().host() + "; exhausted inet socket addresses: " + inetSocketAddresses);}return inetSocketAddresses.get(nextInetSocketAddressIndex++);}/** Returns true if there is another postponed route to try. */private boolean hasNextPostponed() {return !postponedRoutes.isEmpty();}/** Returns the next postponed route to try. */private Route nextPostponed() {return postponedRoutes.remove(0);}
RouteSelector
实现了两级迭代器来提供选择路由的服务。维护接失败的路由的信息,以避免浪费时间去连接一些不可用的路由。
RouteSelector
借助于RouteDatabase
维护失败的路由的信息。/*** Clients should invoke this method when they encounter a connectivity failure on a connection* returned by this route selector.*/public void connectFailed(Route failedRoute, IOException failure) { if (failedRoute.proxy().type() != Proxy.Type.DIRECT && address.proxySelector() != null) { // Tell the proxy selector when we fail to connect on a fresh connection. address.proxySelector().connectFailed( address.url().uri(), failedRoute.proxy().address(), failure); } routeDatabase.failed(failedRoute);}
RouteDatabase
是一个简单的容器:public final class RouteDatabase {private final Set<Route> failedRoutes = new LinkedHashSet<>();/** Records a failure connecting to {@code failedRoute}. */public synchronized void failed(Route failedRoute) { failedRoutes.add(failedRoute);}/** Records success connecting to {@code failedRoute}. */public synchronized void connected(Route route) { failedRoutes.remove(route);}/** Returns true if {@code route} has failed recently and should be avoided. */public synchronized boolean shouldPostpone(Route route) { return failedRoutes.contains(route);}}
代理选择器ProxySelector的实现
在OkHttp3中,
ProxySelector
对象由OkHttpClient维护。
public class OkHttpClient implements Cloneable, Call.Factory {...... final ProxySelector proxySelector; private OkHttpClient(Builder builder) { this.dispatcher = builder.dispatcher; this.proxy = builder.proxy; this.protocols = builder.protocols; this.connectionSpecs = builder.connectionSpecs; this.interceptors = Util.immutableList(builder.interceptors); this.networkInterceptors = Util.immutableList(builder.networkInterceptors); this.proxySelector = builder.proxySelector;...... public ProxySelector proxySelector() { return proxySelector; }...... public Builder() { dispatcher = new Dispatcher(); protocols = DEFAULT_PROTOCOLS; connectionSpecs = DEFAULT_CONNECTION_SPECS; proxySelector = ProxySelector.getDefault();...... Builder(OkHttpClient okHttpClient) { this.dispatcher = okHttpClient.dispatcher; this.proxy = okHttpClient.proxy; this.protocols = okHttpClient.protocols; this.connectionSpecs = okHttpClient.connectionSpecs; this.interceptors.addAll(okHttpClient.interceptors); this.networkInterceptors.addAll(okHttpClient.networkInterceptors); this.proxySelector = okHttpClient.proxySelector;
在创建OkHttpClient时,可以通过为OkHttpClient.Builder设置ProxySelector
来定制ProxySelector
。若没有指定,则使用系统默认的ProxySelector
。OpenJDK 1.8版默认的ProxySelector
为sun.net.spi.DefaultProxySelector
:
public abstract class ProxySelector { /** * The system wide proxy selector that selects the proxy server to * use, if any, when connecting to a remote object referenced by * an URL. * * @see #setDefault(ProxySelector) */ private static ProxySelector theProxySelector; static { try { Class<?> c = Class.forName("sun.net.spi.DefaultProxySelector"); if (c != null && ProxySelector.class.isAssignableFrom(c)) { theProxySelector = (ProxySelector) c.newInstance(); } } catch (Exception e) { theProxySelector = null; } } /** * Gets the system-wide proxy selector. * * @throws SecurityException * If a security manager has been installed and it denies * {@link NetPermission}{@code ("getProxySelector")} * @see #setDefault(ProxySelector) * @return the system-wide {@code ProxySelector} * @since 1.5 */ public static ProxySelector getDefault() { SecurityManager sm = System.getSecurityManager(); if (sm != null) { sm.checkPermission(SecurityConstants.GET_PROXYSELECTOR_PERMISSION); } return theProxySelector; }
在Android平台上,默认ProxySelector
所用的则是另外的实现:
public abstract class ProxySelector { private static ProxySelector defaultSelector = new ProxySelectorImpl(); /** * Returns the default proxy selector, or null if none exists. */ public static ProxySelector getDefault() { return defaultSelector; } /** * Sets the default proxy selector. If {@code selector} is null, the current * proxy selector will be removed. */ public static void setDefault(ProxySelector selector) { defaultSelector = selector; }
Android平台下,默认的ProxySelector
ProxySelectorImpl,其实现 (不同Android版本实现不同,这里以android-6.0.1_r61为例) 如下:
package java.net;import java.io.IOException;import java.util.Collections;import java.util.List;final class ProxySelectorImpl extends ProxySelector { @Override public void connectFailed(URI uri, SocketAddress sa, IOException ioe) { if (uri == null || sa == null || ioe == null) { throw new IllegalArgumentException(); } } @Override public List<Proxy> select(URI uri) { return Collections.singletonList(selectOneProxy(uri)); } private Proxy selectOneProxy(URI uri) { if (uri == null) { throw new IllegalArgumentException("uri == null"); } String scheme = uri.getScheme(); if (scheme == null) { throw new IllegalArgumentException("scheme == null"); } int port = -1; Proxy proxy = null; String nonProxyHostsKey = null; boolean httpProxyOkay = true; if ("http".equalsIgnoreCase(scheme)) { port = 80; nonProxyHostsKey = "http.nonProxyHosts"; proxy = lookupProxy("http.proxyHost", "http.proxyPort", Proxy.Type.HTTP, port); } else if ("https".equalsIgnoreCase(scheme)) { port = 443; nonProxyHostsKey = "https.nonProxyHosts"; // RI doesn't support this proxy = lookupProxy("https.proxyHost", "https.proxyPort", Proxy.Type.HTTP, port); } else if ("ftp".equalsIgnoreCase(scheme)) { port = 80; // not 21 as you might guess nonProxyHostsKey = "ftp.nonProxyHosts"; proxy = lookupProxy("ftp.proxyHost", "ftp.proxyPort", Proxy.Type.HTTP, port); } else if ("socket".equalsIgnoreCase(scheme)) { httpProxyOkay = false; } else { return Proxy.NO_PROXY; } if (nonProxyHostsKey != null && isNonProxyHost(uri.getHost(), System.getProperty(nonProxyHostsKey))) { return Proxy.NO_PROXY; } if (proxy != null) { return proxy; } if (httpProxyOkay) { proxy = lookupProxy("proxyHost", "proxyPort", Proxy.Type.HTTP, port); if (proxy != null) { return proxy; } } proxy = lookupProxy("socksProxyHost", "socksProxyPort", Proxy.Type.SOCKS, 1080); if (proxy != null) { return proxy; } return Proxy.NO_PROXY; } /** * Returns the proxy identified by the {@code hostKey} system property, or * null. */ private Proxy lookupProxy(String hostKey, String portKey, Proxy.Type type, int defaultPort) { String host = System.getProperty(hostKey); if (host == null || host.isEmpty()) { return null; } int port = getSystemPropertyInt(portKey, defaultPort); return new Proxy(type, InetSocketAddress.createUnresolved(host, port)); } private int getSystemPropertyInt(String key, int defaultValue) { String string = System.getProperty(key); if (string != null) { try { return Integer.parseInt(string); } catch (NumberFormatException ignored) { } } return defaultValue; } /** * Returns true if the {@code nonProxyHosts} system property pattern exists * and matches {@code host}. */ private boolean isNonProxyHost(String host, String nonProxyHosts) { if (host == null || nonProxyHosts == null) { return false; } // construct pattern StringBuilder patternBuilder = new StringBuilder(); for (int i = 0; i < nonProxyHosts.length(); i++) { char c = nonProxyHosts.charAt(i); switch (c) { case '.': patternBuilder.append("\\."); break; case '*': patternBuilder.append(".*"); break; default: patternBuilder.append(c); } } // check whether the host is the nonProxyHosts. String pattern = patternBuilder.toString(); return host.matches(pattern); }}
在Android平台上,主要是从系统属性System properties中获取代理服务器的配置信息,这里会过滤掉不能进行代理的主机的访问。
前面我们看到 RouteSelector
通过 Address
提供的Proxy和ProxySelector来收集Proxy信息及连接的目标地址信息。OkHttp3中用 Address
描述建立连接所需的配置信息,包括HTTP服务器的地址,DNS,SocketFactory,Proxy,ProxySelector及TLS所需的一些设施等等:
public final class Address { final HttpUrl url; final Dns dns; final SocketFactory socketFactory; final Authenticator proxyAuthenticator; final List<Protocol> protocols; final List<ConnectionSpec> connectionSpecs; final ProxySelector proxySelector; final Proxy proxy; final SSLSocketFactory sslSocketFactory; final HostnameVerifier hostnameVerifier; final CertificatePinner certificatePinner; public Address(String uriHost, int uriPort, Dns dns, SocketFactory socketFactory, SSLSocketFactory sslSocketFactory, HostnameVerifier hostnameVerifier, CertificatePinner certificatePinner, Authenticator proxyAuthenticator, Proxy proxy, List<Protocol> protocols, List<ConnectionSpec> connectionSpecs, ProxySelector proxySelector) { this.url = new HttpUrl.Builder() .scheme(sslSocketFactory != null ? "https" : "http") .host(uriHost) .port(uriPort) .build(); if (dns == null) throw new NullPointerException("dns == null"); this.dns = dns; if (socketFactory == null) throw new NullPointerException("socketFactory == null"); this.socketFactory = socketFactory; if (proxyAuthenticator == null) { throw new NullPointerException("proxyAuthenticator == null"); } this.proxyAuthenticator = proxyAuthenticator; if (protocols == null) throw new NullPointerException("protocols == null"); this.protocols = Util.immutableList(protocols); if (connectionSpecs == null) throw new NullPointerException("connectionSpecs == null"); this.connectionSpecs = Util.immutableList(connectionSpecs); if (proxySelector == null) throw new NullPointerException("proxySelector == null"); this.proxySelector = proxySelector; this.proxy = proxy; this.sslSocketFactory = sslSocketFactory; this.hostnameVerifier = hostnameVerifier; this.certificatePinner = certificatePinner; } /** * Returns a URL with the hostname and port of the origin server. The path, query, and fragment of * this URL are always empty, since they are not significant for planning a route. */ public HttpUrl url() { return url; } /** Returns the service that will be used to resolve IP addresses for hostnames. */ public Dns dns() { return dns; } /** Returns the socket factory for new connections. */ public SocketFactory socketFactory() { return socketFactory; } /** Returns the client's proxy authenticator. */ public Authenticator proxyAuthenticator() { return proxyAuthenticator; } /** * Returns the protocols the client supports. This method always returns a non-null list that * contains minimally {@link Protocol#HTTP_1_1}. */ public List<Protocol> protocols() { return protocols; } public List<ConnectionSpec> connectionSpecs() { return connectionSpecs; } /** * Returns this address's proxy selector. Only used if the proxy is null. If none of this * selector's proxies are reachable, a direct connection will be attempted. */ public ProxySelector proxySelector() { return proxySelector; } /** * Returns this address's explicitly-specified HTTP proxy, or null to delegate to the {@linkplain * #proxySelector proxy selector}. */ public Proxy proxy() { return proxy; } /** Returns the SSL socket factory, or null if this is not an HTTPS address. */ public SSLSocketFactory sslSocketFactory() { return sslSocketFactory; } /** Returns the hostname verifier, or null if this is not an HTTPS address. */ public HostnameVerifier hostnameVerifier() { return hostnameVerifier; } /** Returns this address's certificate pinner, or null if this is not an HTTPS address. */ public CertificatePinner certificatePinner() { return certificatePinner; }......}
OkHttp3中通过职责链执行HTTP请求。在其中的RetryAndFollowUpInterceptor里创建Address对象时,从OkHttpClient对象获取ProxySelector。Address对象会被用于创建StreamAllocation对象。StreamAllocation在建立连接时,从Address对象中获取ProxySelector以选择路由。
public final class RetryAndFollowUpInterceptor implements Interceptor {...... private Address createAddress(HttpUrl url) { SSLSocketFactory sslSocketFactory = null; HostnameVerifier hostnameVerifier = null; CertificatePinner certificatePinner = null; if (url.isHttps()) { sslSocketFactory = client.sslSocketFactory(); hostnameVerifier = client.hostnameVerifier(); certificatePinner = client.certificatePinner(); } return new Address(url.host(), url.port(), client.dns(), client.socketFactory(), sslSocketFactory, hostnameVerifier, certificatePinner, client.proxyAuthenticator(), client.proxy(), client.protocols(), client.connectionSpecs(), client.proxySelector()); }
在StreamAllocation中,Address对象会被用于创建 RouteSelector
对象:
public final class StreamAllocation {...... public StreamAllocation(ConnectionPool connectionPool, Address address) { this.connectionPool = connectionPool; this.address = address; this.routeSelector = new RouteSelector(address, routeDatabase()); }
代理协议
如我们在 OkHttp3 HTTP请求执行流程分析 中看到的,OkHttp3对HTTP请求是通过Interceptor链来处理的。RetryAndFollowUpInterceptor
创建StreamAllocation
对象,处理http的重定向及出错重试。对后续Interceptor的执行的影响为修改Request并创建StreamAllocation对象。BridgeInterceptor
补全缺失的一些http header。对后续Interceptor的执行的影响主要为修改了Request。CacheInterceptor
处理http缓存。对后续Interceptor的执行的影响为,若缓存中有所需请求的响应,则后续Interceptor不再执行。ConnectInterceptor
借助于前面分配的StreamAllocation
对象建立与服务器之间的连接,并选定交互所用的协议是HTTP 1.1还是HTTP 2。对后续Interceptor的执行的影响为,创建了HttpStream和connection。CallServerInterceptor
作为Interceptor链中的最后一个Interceptor,用于处理IO,与服务器进行数据交换。
在OkHttp3中,收集的路由信息,是在ConnectInterceptor
中建立连接时用到的。ConnectInterceptor
借助于 StreamAllocation
完成整个连接的建立,包括TCP连接建立,代理协议所要求的协商,以及SSL/TLS协议的协商,如ALPN等。我们暂时略过整个连接建立的完整过程,主要关注TCP连接建立及代理协议的协商过程的部分。
StreamAllocation
的findConnection()用来为某次特定的网络请求寻找一个可用的连接。
/** * Returns a connection to host a new stream. This prefers the existing connection if it exists, * then the pool, finally building a new connection. */ private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout, boolean connectionRetryEnabled) throws IOException { Route selectedRoute; synchronized (connectionPool) { if (released) throw new IllegalStateException("released"); if (codec != null) throw new IllegalStateException("codec != null"); if (canceled) throw new IOException("Canceled"); RealConnection allocatedConnection = this.connection; if (allocatedConnection != null && !allocatedConnection.noNewStreams) { return allocatedConnection; } // Attempt to get a connection from the pool. RealConnection pooledConnection = Internal.instance.get(connectionPool, address, this); if (pooledConnection != null) { this.connection = pooledConnection; return pooledConnection; } selectedRoute = route; } if (selectedRoute == null) { selectedRoute = routeSelector.next(); synchronized (connectionPool) { route = selectedRoute; refusedStreamCount = 0; } } RealConnection newConnection = new RealConnection(selectedRoute); acquire(newConnection); synchronized (connectionPool) { Internal.instance.put(connectionPool, newConnection); this.connection = newConnection; if (canceled) throw new IOException("Canceled"); } newConnection.connect(connectTimeout, readTimeout, writeTimeout, address.connectionSpecs(), connectionRetryEnabled); routeDatabase().connected(newConnection.route()); return newConnection; }
OkHttp3中有一套连接池的机制,这里先尝试从连接池中寻找可用的连接,找不到时才会新建连接。新建连接的过程是:
- 选择一个Route;
- 创建
RealConnection
连接对象。 - 将连接对象保存进连接池中。
- 建立连接。
RealConnection
中建立连接的过程是这样的:
public final class RealConnection extends Http2Connection.Listener implements Connection { private final Route route; /** The low-level TCP socket. */ private Socket rawSocket; /** * The application layer socket. Either an {@link SSLSocket} layered over {@link #rawSocket}, or * {@link #rawSocket} itself if this connection does not use SSL. */ public Socket socket; private Handshake handshake; private Protocol protocol; public volatile Http2Connection http2Connection; public int successCount; public BufferedSource source; public BufferedSink sink; public int allocationLimit; public final List<Reference<StreamAllocation>> allocations = new ArrayList<>(); public boolean noNewStreams; public long idleAtNanos = Long.MAX_VALUE; public RealConnection(Route route) { this.route = route; } public void connect(int connectTimeout, int readTimeout, int writeTimeout, List<ConnectionSpec> connectionSpecs, boolean connectionRetryEnabled) { if (protocol != null) throw new IllegalStateException("already connected"); RouteException routeException = null; ConnectionSpecSelector connectionSpecSelector = new ConnectionSpecSelector(connectionSpecs); if (route.address().sslSocketFactory() == null) { if (!connectionSpecs.contains(ConnectionSpec.CLEARTEXT)) { throw new RouteException(new UnknownServiceException( "CLEARTEXT communication not enabled for client")); } String host = route.address().url().host(); if (!Platform.get().isCleartextTrafficPermitted(host)) { throw new RouteException(new UnknownServiceException( "CLEARTEXT communication to " + host + " not permitted by network security policy")); } } while (protocol == null) { try { if (route.requiresTunnel()) { buildTunneledConnection(connectTimeout, readTimeout, writeTimeout, connectionSpecSelector); } else { buildConnection(connectTimeout, readTimeout, writeTimeout, connectionSpecSelector); } } catch (IOException e) { closeQuietly(socket); closeQuietly(rawSocket); socket = null; rawSocket = null; source = null; sink = null; handshake = null; protocol = null; if (routeException == null) { routeException = new RouteException(e); } else { routeException.addConnectException(e); } if (!connectionRetryEnabled || !connectionSpecSelector.connectionFailed(e)) { throw routeException; } } } }
在这个方法中,SSLSocketFactory为空,也就是要求请求/响应明文传输时,先做安全性检查,以确认系统允许明文传输,允许以请求的域名做明文传输。
然后根据路由的具体情况,执行不同的连接建立过程。对于需要创建隧道连接的路由,执行buildTunneledConnection(),对于其它情况,则执行buildConnection()。
判断是否要建立隧道连接的依据是代理的类型,以及连接的类型:
/** * Returns true if this route tunnels HTTPS through an HTTP proxy. See <a * href="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817, Section 5.2</a>. */ public boolean requiresTunnel() { return address.sslSocketFactory != null && proxy.type() == Proxy.Type.HTTP; }
如果是HTTP代理,且请求建立SSL/TLS加密通道 (http/1.1的https和http2) ,则需要建立隧道连接。其它情形不需要建立隧道连接。
非隧道连接的建立
非隧道连接的建立过程为:
/** Does all the work necessary to build a full HTTP or HTTPS connection on a raw socket. */ private void buildConnection(int connectTimeout, int readTimeout, int writeTimeout, ConnectionSpecSelector connectionSpecSelector) throws IOException { connectSocket(connectTimeout, readTimeout); establishProtocol(readTimeout, writeTimeout, connectionSpecSelector); } private void connectSocket(int connectTimeout, int readTimeout) throws IOException { Proxy proxy = route.proxy(); Address address = route.address(); rawSocket = proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.HTTP ? address.socketFactory().createSocket() : new Socket(proxy); rawSocket.setSoTimeout(readTimeout); try { Platform.get().connectSocket(rawSocket, route.socketAddress(), connectTimeout); } catch (ConnectException e) { throw new ConnectException("Failed to connect to " + route.socketAddress()); } source = Okio.buffer(Okio.source(rawSocket)); sink = Okio.buffer(Okio.sink(rawSocket)); }
有 3 种情况需要建立非隧道连接:
- 无代理。
- 明文的HTTP代理。
- SOCKS代理。
非隧道连接的建立过程为建立TCP连接,然后在需要时完成SSL/TLS的握手及HTTP/2的握手建立Protocol。建立TCP连接的过程为:
- 创建Socket。非SOCKS代理的情况下,通过SocketFactory创建;在SOCKS代理则传入proxy手动new一个出来。
- 为Socket设置读超时。
- 完成特定于平台的连接建立。
- 创建用语IO的source和sink。
AndroidPlatform
的 connectSocket()
是这样的:
@Override public void connectSocket(Socket socket, InetSocketAddress address, int connectTimeout) throws IOException { try { socket.connect(address, connectTimeout); } catch (AssertionError e) { if (Util.isAndroidGetsocknameError(e)) throw new IOException(e); throw e; } catch (SecurityException e) { // Before android 4.3, socket.connect could throw a SecurityException // if opening a socket resulted in an EACCES error. IOException ioException = new IOException("Exception in connect"); ioException.initCause(e); throw ioException; } }
设置了SOCKS代理的情况下,仅有的特别之处在于,是通过传入proxy手动创建的Socket。route的socketAddress包含着目标HTTP服务器的域名。由此可见SOCKS协议的处理,主要是在Java标准库的 java.net.Socket
中处理的。对于外界而言,就好像是于HTTP服务器直接建立连接一样,因为连接时传入的地址都是HTTP服务器的域名。
而对于明文HTTP代理的情况下,这里没有任何特殊的处理。route的socketAddress包含着代理服务器的IP地址。HTTP代理自身会根据请求及响应的实际内容,建立与HTTP服务器的TCP连接,并转发数据。猜测HTTP代理服务器是根据HTTP请求中的"Host"等header内容来确认HTTP服务器地址的。
暂时先略过对建立协议过程的分析。
HTTP代理的隧道连接
buildTunneledConnection()用于建立隧道连接:
/** * Does all the work to build an HTTPS connection over a proxy tunnel. The catch here is that a * proxy server can issue an auth challenge and then close the connection. */ private void buildTunneledConnection(int connectTimeout, int readTimeout, int writeTimeout, ConnectionSpecSelector connectionSpecSelector) throws IOException { Request tunnelRequest = createTunnelRequest(); HttpUrl url = tunnelRequest.url(); int attemptedConnections = 0; int maxAttempts = 21; while (true) { if (++attemptedConnections > maxAttempts) { throw new ProtocolException("Too many tunnel connections attempted: " + maxAttempts); } connectSocket(connectTimeout, readTimeout); tunnelRequest = createTunnel(readTimeout, writeTimeout, tunnelRequest, url); if (tunnelRequest == null) break; // Tunnel successfully created. // The proxy decided to close the connection after an auth challenge. We need to create a new // connection, but this time with the auth credentials. closeQuietly(rawSocket); rawSocket = null; sink = null; source = null; } establishProtocol(readTimeout, writeTimeout, connectionSpecSelector); }
这里主要是两个过程:
- 建立隧道连接。
- 建立Protocol。
建立隧道连接的过程又分为几个步骤:
- 创建隧道请求
- 建立Socket连接
- 发送请求建立隧道
隧道请求是一个常规的HTTP请求,只是请求的内容有点特殊。最初创建的隧道请求如:
/** * Returns a request that creates a TLS tunnel via an HTTP proxy. Everything in the tunnel request * is sent unencrypted to the proxy server, so tunnels include only the minimum set of headers. * This avoids sending potentially sensitive data like HTTP cookies to the proxy unencrypted. */ private Request createTunnelRequest() { return new Request.Builder() .url(route.address().url()) .header("Host", Util.hostHeader(route.address().url(), true)) .header("Proxy-Connection", "Keep-Alive") .header("User-Agent", Version.userAgent()) // For HTTP/1.0 proxies like Squid. .build(); }
一个隧道请求的例子如下:
请求的"Host" header中包含了目标HTTP服务器的域名。建立socket连接的过程这里不再赘述。
创建隧道的过程是这样子的:
/** * To make an HTTPS connection over an HTTP proxy, send an unencrypted CONNECT request to create * the proxy connection. This may need to be retried if the proxy requires authorization. */ private Request createTunnel(int readTimeout, int writeTimeout, Request tunnelRequest, HttpUrl url) throws IOException { // Make an SSL Tunnel on the first message pair of each SSL + proxy connection. String requestLine = "CONNECT " + Util.hostHeader(url, true) + " HTTP/1.1"; while (true) { Http1Codec tunnelConnection = new Http1Codec(null, null, source, sink); source.timeout().timeout(readTimeout, MILLISECONDS); sink.timeout().timeout(writeTimeout, MILLISECONDS); tunnelConnection.writeRequest(tunnelRequest.headers(), requestLine); tunnelConnection.finishRequest(); Response response = tunnelConnection.readResponse().request(tunnelRequest).build(); // The response body from a CONNECT should be empty, but if it is not then we should consume // it before proceeding. long contentLength = HttpHeaders.contentLength(response); if (contentLength == -1L) { contentLength = 0L; } Source body = tunnelConnection.newFixedLengthSource(contentLength); Util.skipAll(body, Integer.MAX_VALUE, TimeUnit.MILLISECONDS); body.close(); switch (response.code()) { case HTTP_OK: // Assume the server won't send a TLS ServerHello until we send a TLS ClientHello. If // that happens, then we will have buffered bytes that are needed by the SSLSocket! // This check is imperfect: it doesn't tell us whether a handshake will succeed, just // that it will almost certainly fail because the proxy has sent unexpected data. if (!source.buffer().exhausted() || !sink.buffer().exhausted()) { throw new IOException("TLS tunnel buffered too many bytes!"); } return null; case HTTP_PROXY_AUTH: tunnelRequest = route.address().proxyAuthenticator().authenticate(route, response); if (tunnelRequest == null) throw new IOException("Failed to authenticate with proxy"); if ("close".equalsIgnoreCase(response.header("Connection"))) { return tunnelRequest; } break; default: throw new IOException( "Unexpected response code for CONNECT: " + response.code()); } } }
在前面创建的TCP连接之上,完成与代理服务器的HTTP请求/响应交互。请求的内容类似下面这样:
"CONNECT m.taobao.com:443 HTTP/1.1"
这里可能会根据HTTP代理是否需要认证而有多次HTTP请求/响应交互。
总结一下OkHttp3中代理相关的处理:
- 没有设置代理的情况下,直接与HTTP服务器建立TCP连接,然后进行HTTP请求/响应的交互。
- 设置了SOCKS代理的情况下,创建Socket时,为其传入proxy,连接时还是以HTTP服务器为目标地址。在标准库的Socket中完成SOCKS协议相关的处理。此时基本上感知不到代理的存在。
- 设置了HTTP代理时的HTTP请求,与HTTP代理服务器建立TCP连接。HTTP代理服务器解析HTTP请求/响应的内容,并根据其中的信息来完成数据的转发。也就是说,如果HTTP请求中不包含"Host" header,则有可能在设置了HTTP代理的情况下无法与HTTP服务器建立连接。
- 设置了HTTP代理时的HTTPS/HTTP2请求,与HTTP服务器建立通过HTTP代理的隧道连接。HTTP代理不再解析传输的数据,仅仅完成数据转发的功能。此时HTTP代理的功能退化为如同SOCKS代理类似。
- 设置了代理时,HTTP服务器的域名解析会被交给代理服务器执行。其中设置了HTTP代理时,会对HTTP代理的域名做域名解析。
关于HTTP代理的更多内容,可以参考HTTP 代理原理及实现(一)。
OkHttp3中代理相关的处理大体如此。
- OkHttp3中的代理与路由
- Flask 中的路由与反向路由
- Flask 中的路由与反向路由
- 路由、NAT与代理的区别
- Glide与OkHttp3集成
- OKHttp3初始化与配置
- java中的静态代理与动态代理
- java中的静态代理与动态代理
- java中的静态代理与动态代理
- Android中的OkHttp3简单用法.
- okhttp3在Fragment中的使用
- OSPF路由协议中的邻居与邻接
- java中的代理 静态代理与动态代理
- C#中的事件与代理
- C#中的代理与事件
- C#中的代理与事件
- IOS中的协议与代理
- C#中的代理与事件
- 用FlatBuffers提升Android平台上Facebook的性能
- ElasticSearch-创建索引CreateIndex
- caffe 主要是讲一下caffe对不同的数据类型的处理(二)
- spring学习笔记 -- day11 spring中的事务控制
- java中成员变量和局部变量在内存中的分配
- OkHttp3中的代理与路由
- 使用QUIC
- RestTemplate
- Cronet android 设计与实现分析——备选服务机制
- 生产者/消费者模型改进版 ——队列
- OkHttp3的连接池及连接建立过程分析
- 非对称加密与证书
- Maven全局排除某引用的一种方式
- HTTP/2 流量调试