varnish.md

varnish

webcache
命中率：hit/（hit+miss）
命中次数/（命中次数+未命中）

文档命中率：从文档个数进行衡量：字节命中率：从内容大小进行衡量；

缓存处理步骤
接收请求-->解析请求（提取请求的URL及各种首部）->查询缓存-->新鲜度检测-->创建响应报文-->发送响应-->记录日志

varnish如何存储缓存对象：
file：单个文件；不支持持久机制：
malloc：内存；
persistent：基于文件的持久存储；

配置varnish的三种应用：
1、varnishd应用程序的命令行参数；
监听的socket，使用的存储类型等等；额外的配置参数；
-p param=value
-r param，param，..：设定只读参数列表；
/etc/varnish/varnish.params

2、-p选项指明的参数：
运行时参数：
也可在程序运行中，通过其CLI进行配置：

3、vc1：配置缓存系统的缓存机制；
通过vc1配置文件进行配置：
先编译，后应用：
依赖于c编译器；

安装 varnish

yum install varnish -y
cat /usr/lib/systemd/system/varnish.service
yum info jemalloc
man varnished

vim varnish.params
VARNISH_STORAGE="malloc,256M"

vim default.vcl
.host="192.168.100.102"  webservers
.port="80"
systemctl restart varnish

ss -tnl 6081 6082

varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082
help
ping
status
vcl.list
vcl.load testname default.vcl
vcl.list
vcl.use testname
vcl.list
vcl.discard testname
vcl.list
vcl.show boot

param.show
param.set thread_pools 4
param.show -l 
param.show thread_pools

panic.show
storage.list
backend.list
ban.list
quit

可使用telnet远程登陆
#log
varnishlog  
vcrnishnsca
#top
varnishtop
#statistics
varnishstat

示例一：测试访问缓存服务器，缓存是否命中

varnish server

cat /etc/varnish/default.vcl
#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "192.168.100.102";
    .port = "80";
}

sub vcl_recv {
    # Happens before we check if we have this in cache already.
    #
    # Typically you clean up the request here, removing cookies you don't need,
    # rewriting the request, etc.
    if(req.method == "PRI"){
    /* we do not support SPDY or HTTP/2.0*/
    return(synth(405)); 
}
if(req.method != "GET" &&
    req.method != "HEAD" && 
    req.method != "PUT" &&
    req.method != "POST" &&
    req.method != "TRACE" &&
    req.method != "OPTIONS" &&
    req.method != "DELETE"){
    /*=Non-RFC2616 or CONNECT which is weird.*/
    return(pipe);
}
if(req.method !="GET" && req.method != "HEAD"){
    /* We only deal with GET and HEAD by default */
    return(pass); 
}
if(req.http.Authorization || req.http.Cookie){
    /* Not cacheable by default */
    return(pass);
}
	return(hash);
}

sub vcl_backend_response {
    # Happens after we have read the response headers from the backend.
    #
    # Here you clean the response headers, removing silly Set-Cookie headers
    # and other mistakes your backend does.
}

sub vcl_deliver {
    # Happens when we have all the pieces we need, and are about to send the
    # response to the client.
    #
    # You can do accounting or modifying the final object here.
    if (obj.hits>0) {
	set resp.http.X-Cache="HIT";
	} else {
	set resp.http.X-Cache="MISS";	
}
}

web1，web2

for i in {1..10}; do echo "this is web1 page $i" > /var/www/html/test$i.html; done
systemctl restart httpd
systemctl enable httpd
for i in {1..10}; do echo "this is web2 page $i" > /var/www/html/test$i.html; done
systemctl restart httpd
systemctl enable httpd

客户端测试

for i in {1..2};do curl -I http://192.168.100.101:6081/test1.html;done|grep X-Cache

示例二：

强制对某资源的请求，不检查缓存

/admin
/login

#访问以 http://localhost/[login或/admin]（不区分大小写）不检查缓存
sub vcl_recv{
if (req.url ~ "(?i)^/login" || req.url ~ "(?i)^/admin" ){
    return(pass);
}
}

webs

echo admin web2s > /var/www/html/admin/index.html

客户端测试

curl http://192.168.100.101:6081/admin/index.html -I|grep MISS

示例三

对特定类型的资源取消其私有的COOKICE标识,并强行设定其可以varnish缓存的时长

#访问JPG 3600秒内都命中 HIT 
sub vcl_backend_response {
   if (beresp.http.cache-control !~ "s-maxage") {
  	 if (bereq.url ~ "(?i).txt$") {
        set beresp.ttl = 3600s;
        unset beresp.http.Set-Cookie;
   	 }
	if (bereq.url ~ "(?i).jpg$") {
        set beresp.ttl = 600s;
        unset beresp.http.Set-Cookie;
   	 }
	}
}

客户端测试

curl http://192.168.100.101:6081/1.txt -I

示例四：

可以做动静分离

官方配置示例https://varnish-cache.org/trac/wiki/VCLExamples

backend server的定义

.host：BE主机的IP：
.port：BE主机监听的PORT:

.probe：对BE做健康状态检测
.max_connections：并发连接最大数量

后端主机的健康状态检测方式

.url：判定BE健康与否要请求的url；
.expected_response：期望响应状态码；默认为200；

backend web1 {
    .host = "192.168.200.101";
    .port = "80";
	.probe = {
        .url = "/test1.html";
	}
}
backend web2 {
    .host = "192.168.200.102";
    .port = "80";
	.probe = {
        .url = "/test1.html";
	}
}

sub vcl_recv {  #此处只能出现一次，此有此项，将下面内容附加到后面即可
    if (req.url ~ "(?i).(jpg|pn|gif|txt)$") {
        set req.backend_hint = web1;
    } else {
        set req.backend_hint = web2;
    }
}

客户端测试

curl http://192.168.100.101:6081/1.txt -I
curl http://192.168.100.101:6081/test1.html -I

示例五

负载均衡算法
fallback,random,round_robin,hash

#轮询是对单个资源做round_robin
import directors;
sub vcl_init {
    new mycluster = directors.round_robin();
    mycluster.add_backend(web1);
    mycluster.add_backend(web2);
}

sub vcl_recv {
	set req.backend_hint = mycluster.backend();
}

客户端测试

curl http://192.168.100.101:6081/test1.html
curl http://192.168.100.101:6081/test2.html

若要更改varnish端口

vim /etc/varnish/varnish.params
VARNISH_LISTEN_PORT=80

systemctl restart varnish

conf

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend web1{
    .host = "192.168.100.101";
    .port = "80";
    .probe = {
	.url = "/test1.html";
	}
}
backend web2 {
    .host = "192.168.100.102";
    .port = "80";
    .probe = {
	.url = "/test1.html";
	}
}
import directors;
sub vcl_init {
    new mycluster = directors.round_robin();
    mycluster.add_backend(web1);
    mycluster.add_backend(web2);
}
sub vcl_recv {
    # Happens before we check if we have this in cache already.
    #
    # Typically you clean up the request here, removing cookies you don't need,
    # rewriting the request, etc.
    set req.backend_hint = mycluster.backend();
    #    if (req.url ~ "(?i).(jpg|png|gif|txt)$") {
    #    set req.backend_hint = web1;
    #} else {
    #    set req.backend_hint = web2;
    #}
    if (req.url ~ "(?i)^/login" || req.url ~ "(?i)^/admin" ){
    return(pass);
}
    if(req.method == "PRI"){
    /* we do not support SPDY or HTTP/2.0*/
    return(synth(405)); 
}
if(req.method != "GET" &&
    req.method != "HEAD" && 
    req.method != "PUT" &&
    req.method != "POST" &&
    req.method != "TRACE" &&
    req.method != "OPTIONS" &&
    req.method != "DELETE"){
    /*=Non-RFC2616 or CONNECT which is weird.*/
    return(pipe);
}
if(req.method !="GET" && req.method != "HEAD"){
    /* We only deal with GET and HEAD by default */
    return(pass); 
}
if(req.http.Authorization || req.http.Cookie){
    /* Not cacheable by default */
    return(pass);
}
	return(hash);
}

sub vcl_backend_response {
    # Happens after we have read the response headers from the backend.
    #
    # Here you clean the response headers, removing silly Set-Cookie headers
    # and other mistakes your backend does.
   if (beresp.http.cache-control !~ "s-maxage") {
    if (bereq.url ~ "(?i).txt$") {
        set beresp.ttl = 3600s;
        unset beresp.http.Set-Cookie;
    }
    if (bereq.url ~ "(?i).css$") {
        set beresp.ttl = 600s;
        unset beresp.http.Set-Cookie;
    }
}
}

sub vcl_deliver {
    # Happens when we have all the pieces we need, and are about to send the
    # response to the client.
    #
    # You can do accounting or modifying the final object here.
    if (obj.hits>0) {
	set resp.http.X-Cache="HIT";
	} else {
	set resp.http.X-Cache="MISS";	
}
}