varnish

来源：互联网发布：centos 文档编辑：程序博客网时间：2024/04/27 22:30

varnish是一款专业web缓存服务器，对I/O要求特别高，和nginx不同的是，数据缓存在内存中，因此性能非常好，而nginx是存储在levels=1:2:2....这样的层级结构的文件中，一旦层级量大会非常占用inode，而且nginx的侧重点是反代，不是缓存。

有状态和无状态的解释参考：http://blog.sina.com.cn/s/blog_5a15b7d10101h4sk.html

可以做缓存的内容：数据库，流媒体，页面文件，图片，css/html....

varnish的功能：

一套缓存系统，在CDN中扮演web缓存的角色，性能和响应时间更优越，语法简单，可以实现很多定制化，而HAProxy无缓存
反代是最基本的功能，自然具备
可以实现动静分离
可以实现负载均衡

一次完整的请求缓存的工作流程：接收请求 --> 解析请求（提取请求的URL及各种首部）--> 查询缓存 --> 新鲜度检测 --> 创建响应报文 --> 发送响应 --> 记录日志

新鲜度检测机制：
过期日期：
HTTP/1.0 Expires
Expires:Thu, 04 Jun 2015 23:38:18 GMT
HTTP/1.1 Cache-Control: max-age
Cache-Control:max-age=600

有效性再验正：revalidate
如果原始内容未改变，则仅响应首部（不附带body部分），响应码304 （Not Modified）
如果原始内容发生改变，则正常响应，响应码200；
如果原始内容消失，则响应404，此时缓存中的cache object也应该被删除；

条件式请求首部：经典搭配：If-Modified-Since+If-None-Match
If-Modified-Since：基于请求内容的时间戳作验正；
If-Unmodified-Since
If-Match：
If-None-Match：
Etag: faiy89345

host1:192.168.1.3

host2:192.168.1.4

host1安装参考文档：https://www.varnish-cache.org/installation/redhat

For RHEL 7 and compatible distributions, use:

yum install epel-release
rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.0.el7.rpm
yum install varnish

配置varnish本身属性的配置文件：

~]# vim /etc/varnish/varnish.params

#VARNISH_STORAGE="file,/var/lib/varnish/varnish_storage.bin,1G"
VARNISH_STORAGE="malloc,256M"

///总共有三种缓存机制：file、malloc、persistent，file是存在单个文件中，适合存储量大的缓存，而它的元数据（key）是存在内存中的，不支持持久服务，一旦重启服务就没有了；malloc是存在内存中，性能好，但是内存空间有限，并且时间久了，会产生内存碎片，因此不建议使用太大的空间；而persistent是可以将file信息缓存在磁盘文件中的，但是目前还在测试阶段。如果条件允许，建议使用file机制+SSD

启动服务：

systemctl start varnish.service

///监听的服务端口 6081/tcp

varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082

/// 监听的管理接口 port　　6082/tcp

实时配置

~]#varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082

vcl.load first default.vcl
200
VCL compiled.
vcl.use first
200
VCL 'first' now active
vcl.list
200
available 0 boot
active 0 first

param.show

varnish> param.set thread_pools 4

host2:

yum install httpd -y

for i in {1..10};do echo "welcome page $1" > /var/www/html/test$i.html;done

systemctl start httpd

浏览器测试：http://192.168.1.3:6081/test1.html

示范缓存简单配置：

官方文档：https://www.varnish-cache.org/docs/4.0/reference/vcl.html#varnish-configuration-language

# cat default.vcl
#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend default {
.host = "192.168.1.4";
.port = "80";
}

sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
if (req.method == "PRI") {
/* We do not support SPDY or HTTP/2.0 */
return (synth(405));
}

if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}

if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
return (hash);
}

sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
}

sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
# You can do accounting or modifying the final object here.
if (obj.hits>0) {
set resp.http.X-Cache = "HIT from" + server.ip; ///向相应给客户端的报文添加自定义首部X-Cache
} else {
set resp.http.X-Cahce = "MISS from" + server.ip;
}
}

测试：

host1:

varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082

varnish> vcl.load test5 default.vcl
200
VCL compiled.
vcl.use test5
200
VCL 'test5' now active

host3：

[root@localhost ~]# curl -I http://192.168.1.3:6081/test4.html
HTTP/1.1 200 OK
Date: Mon, 11 Apr 2016 14:12:47 GMT
Server: Apache/2.4.6 (CentOS)
Last-Modified: Mon, 11 Apr 2016 09:12:39 GMT
ETag: "f-53031f248496b"
Content-Length: 15
Content-Type: text/html; charset=UTF-8
X-Varnish: 131076
Age: 0
Via: 1.1 varnish-v4
X-Cahce: MISS from192.168.1.3 ///首次访问没命中
Connection: keep-alive

[root@localhost ~]# curl -I http://192.168.1.3:6081/test4.html
HTTP/1.1 200 OK
Date: Mon, 11 Apr 2016 14:12:47 GMT
Server: Apache/2.4.6 (CentOS)
Last-Modified: Mon, 11 Apr 2016 09:12:39 GMT
ETag: "f-53031f248496b"
Content-Length: 15
Content-Type: text/html; charset=UTF-8
X-Varnish: 131079 131077
Age: 3
Via: 1.1 varnish-v4
X-Cache: HIT from192.168.1.3 ///第二次访问命中
Connection: keep-alive

支持虚拟机

sub vcl_recv {

if (req.host == "www.a.com" ) {

.......

}

不缓存内容，不缓存登陆的账号密码，用return(pass)，送往后端

sub vcl_recv {

if (req.url ~ "(?i)\.test7.html$") {
return (pass);

}
if ( req.url ~ "(?i)^/admin" || req.url ~ "(?i)^/login" ) {
return(pass);
}

不缓存特定资源私有cookie标识，强制设置varnish缓存时长，s-maxage是公共缓存时长，而max-age公共或私有的缓存时长

官方配置示例：https://www.varnish-cache.org/trac/wiki/VCLExamples

sub vcl_backend_response {
# and other mistakes your backend does.
if (bereq.http.cache-control !~ "s-maxage") {
if (bereq.url ~ "(?i)\.(jpg|png|gif$)") {
set beresp.ttl=3600s; ///设置varnish缓存时长，记得要带单位
unset beresp.http.Set-Cookie; ///取消特定资源缓存私有cookie标识
}
if (bereq.url ~ "(?i)\.css$") {
set beresp.ttl=600s;
unset beresp.http.Set-Cookie;
}
}

对静态网页和图片实现分离

backend webserver1 {
.host="192.168.1.2";
.port="80";
.probe={
.url="/test1.html"; ///健康状态检测，检测页面为test1.html
}
}
backend imageserver1 {
.host="192.168.1.4";
.port="80";
.probe={
.url="/test1.html";
}
}

sub vcl_recv {

if (req.url ~ "(?i)\.(jpg|gif|png|jpeg)$") {
set req.backend_hint=imageserver1;
}
else {
set req.backend_hint=webserver1;
}

}

可对backend server进行查看

backend.lvcl.load test6 default.vcl
200
VCL compiled.
vcl.use test6
200
VCL 'test6' now active

backend.list

200
Backend name Refs Admin Probe
default(127.0.0.1,,8080) 1 probe Healthy (no probe)
default(192.168.1.4,,80) 6 probe Healthy (no probe)
webserver1(192.168.1.2,,80) 3 probe Healthy 8/8
imageserver1(192.168.1.4,,80) 3 probe Healthy 8/8

对后端主机进行负载均衡：

backend webserver1 {
.host="192.168.1.2";
.port="80";
.probe={
.url="/test1.html";
}
}
backend imageserver1 {
.host="192.168.1.4";
.port="80";
.probe={
.url="/test1.html";
}
}

import directors; ///一定要指定，指定可以调度，可以调度的算法有fallback，random，round_robin，hash

///特别说明：round_robin跟以往的轮询不同，指的是对不同的资源轮询，而不是对同一个资源轮询

sub vcl_init {
new cluster1=directors.round_robin();
cluster1.add_backend(webserver1); ///由于是round_robin，因此不能指定权重
cluster1.add_backend(imageserver1);
}

sub vcl_recv {
set req.backend_hint=cluster1.backend();

}

测试：由于round_robin跟以往的轮询不同，指的是对不同的资源轮询，而不是对同一个资源轮询，因此

对不同的资源访问，才可以看到跳往不同的backend server，会发现回应的内容来自不同的主机

http://192.168.1.3:6081/test3.html

http://192.168.1.3:6081/test4.html

0 0