[varnish]Varnish安装部署

简述:Varnish是一款高性能的开源HTTP加速器,挪威最大的在线报纸 Verdens Gang 使用3台Varnish代替了原来的12台Squid,性能比以前更好。

系统环境:
CentOS release 5.5 (Final) 64-bit

所需软件:
varnish-2.1.4.tar.gz

Varnish官方网站:
http://www.varnish-cache.org/

安装前准备:

创建apache用户和组,以及Varnish缓存文件存放目录(/var/vcache):

/usr/sbin/groupadd apache -g 48

/usr/sbin/useradd -u 48 -g apache apache

mkdir -p /var/vcache

chmod +w /var/vcache

chown -R apache:apache /var/vcache
创建Varnish日志目录(/var/logs/):
mkdir -p /var/logs

chmod +w /var/logs

chown -R apache:apache /var/logs
安装:
wget http://repo.varnish-cache.org/source/varnish-2.1.4.tar.gz

tar zxvf varnish-2.1.4.tar.gz

cd varnish-2.1.4

./configure -prefix=/usr/local/varnish

make

make install

cd ..
配置:
默认配置文件样板:
/usr/local/varnish/etc/varnish/default.vcl
cd /usr/local/varnish/etc/varnish/
cp default.vcl elain_vcl.conf
vi elain_vcl.conf

#############################
backend www {

.host = “www.elain.org”;

.port = “80”;

}

sub vcl_recv {

if (req.restarts == 0) {

if (req.http.x-forwarded-for) {

set req.http.X-Forwarded-For =

req.http.X-Forwarded-For “, “ client.ip;

} else {

set req.http.X-Forwarded-For = client.ip;

}

}

if (req.request != “GET” &&

req.request != “HEAD” &&

req.request != “PUT” &&

req.request != “POST” &&

req.request != “TRACE” &&

req.request != “OPTIONS” &&

req.request != “DELETE”) {

/ Non-RFC2616 or CONNECT which is weird. /

return (pipe);

}

if (req.request != “GET” && req.request != “HEAD”) {

/ We only deal with GET and HEAD by default /

return (pass);

}

if (req.http.Authorization || req.http.Cookie) {

/ Not cacheable by default /

return (pass);

}

return (lookup);

}

sub vcl_pipe {

return (pipe);

}

sub vcl_pass {

return (pass);

}

sub vcl_hash {

set req.hash += req.url;

if (req.http.host) {

set req.hash += req.http.host;

} else {

set req.hash += server.ip;

}

return (hash);

}

sub vcl_hit {

if (!obj.cacheable) {

return (pass);

}

return (deliver);

}

sub vcl_miss {

return (fetch);

}

sub vcl_fetch {

if (!beresp.cacheable) {

return (pass);

}

if (beresp.http.Set-Cookie) {

return (pass);

}

return (deliver);

}

sub vcl_deliver {

return (deliver);

}

sub vcl_error {

set obj.http.Content-Type = “text/html; charset=utf-8”;

synthetic {“

<?xml version=”1.0” encoding=”utf-8”?>

<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”

http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">





“} obj.status “ “ obj.response {“





Error “} obj.status “ “ obj.response {“



“} obj.response {“



Guru Meditation:



XID: “} req.xid {“






Varnish cache server





“};

return (deliver);

}

###################################
配置文件解释:
(1)、Varnish通过反向代理请求后端IP为1.0.0.121,端口为80的web服务器;
(2)、Varnish允许localhost、127.0.0.1、10.0.0.*三个来源IP通过PURGE方法清除缓存;
(3)、Varnish对域名为www.elain.org的请求进行处理,非www.elain.org域名的请求则返回“elain Cache Server”;
(4)、Varnish对HTTP协议中的GET、HEAD请求进行缓存,对POST请求透过,让其直接访问后端Web服务器。之所以这样配置,是因为POST请求一般是发送数据给服务器的,需要服务器接收、处理,所以不缓存;
(5)、Varnish对以.txt和.js结尾的URL缓存时间设置1小时,对其他的URL缓存时间设置为30天。
启动Varnish
ulimit -SHn 65535

/usr/local/varnish/sbin/varnishd -f /usr/local/varnish/etc/varnish/elain_vcl.conf -a /var/vcache -s malloc,1G -u apache -g apache -T 127.0.0.1:2000 -a 0.0.0.0:8080
注:通常我们apache或nginx等 WEB 会使用8080端口,这样 我们就把上面的启动参数选项-a去掉,让其默认使用80端口(也就是在配置文件里的端口)

启动varnishncsa用来将Varnish访问日志写入日志文件:

/usr/local/varnish/bin/varnishncsa -n /var/vcache -w /var/logs/varnish.log &

停止Varnish
pkill varnish

配置开机自动启动Varnish
vi /etc/rc.local
在末尾增加以下内容:

ulimit -SHn 65535

/usr/local/varnish/sbin/varnishd -f /usr/local/varnish/etc/varnish/elain_vcl.conf -n /var/vcache -s malloc,1G -u apache -g apache -T 127.0.0.1:2000

/usr/local/varnish/bin/varnishncsa -n /var/vcache -w /var/logs/youvideo.log &
优化Linux内核参数
vi /etc/sysctl.conf
在末尾增加以下内容:
net.ipv4.tcp_fin_timeout = 30

net.ipv4.tcp_keepalive_time = 300

net.ipv4.tcp_syncookies = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_tw_recycle = 1

net.ipv4.ip_local_port_range = 5000 65000
sysctl -p

管理Varnish:
1、查看Varnish服务器连接数与命中率:
/usr/local/varnish/bin/varnishstat

2、通过Varnish管理端口进行管理:
用help看看可以使用哪些Varnish命令:
/usr/local/varnish/bin/varnishadm -T 127.0.0.1:2000 help

[root@postfix varnish]# /usr/local/varnish/bin/varnishadm -T 127.0.0.1:2000 help
help [command]
ping [timestamp]
auth response
quit
banner
status
start
stop
stats
vcl.load
vcl.inline
vcl.use
vcl.discard
vcl.list
vcl.show
param.show [-l] []
param.set
purge.url
purge [&& ]…
purge.list

3、通过Varnish管理端口,使用正则表达式批量清除缓存:
(1)、例:清除类似http://www.elain.org/tmp/aa.html的URL地址):
/usr/local/varnish/bin/varnishadm -T 127.0.0.1:2000 url.purge /tmp/

(2)、例:清除类似http://www.elain.org/dl 的URL地址:
/usr/local/varnish/bin/varnishadm -T 127.0.0.1:2000 url.purge w*$

(3)、例:清除所有缓存:
/usr/local/varnish/bin/varnishadm -T 127.0.0.1:2000 url.purge *$

每天0点运行,按天切割Varnish日志,生成一个压缩文件,同时删除上个月旧日志的脚本(/var/logs/cutlog.sh):
/var/logs/cutlog.sh文件内容如下:

#!/bin/sh

This script run at 00:00

date=$(date -d “yesterday” +”%Y-%m-%d”)

pkill -9 varnishncsa

mv /var/logs/youvideo.log /var/logs/${date}.log

/usr/local/varnish/bin/varnishncsa -n /var/vcache -w /var/logs/youvideo.log &

mkdir -p /var/logs/youvideo/

gzip -c /var/logs/${date}.log > /var/logs/youvideo/${date}.log.gz

rm -f /var/logs/${date}.log

rm -f /var/logs/youvideo/$(date -d “-1 month” +”%Y-%m*”).log.gz
设置在每天00:00定时执行:
crontab -e

0 0 * /bin/sh /var/logs/cutlog.sh

配置实例:

#Example 1 - manipulating headers

#Lets say we want to remove the cookie for all objects in the /static directory of our web server::

sub vcl_recv {

if (req.url ~ “^/images”) {

unset req.http.cookie;

}

}
Now, when the request is handled to the backend server there will be no cookie header. The interesting line is the one with the if-statement. It matches the URL, taken from the request object, and matches it against the regular expression. Note the match operator. If it matches the Cookie: header of the request is unset (deleted).

#Example 2 - manipulating beresp

#Here we override the TTL of a object comming from the backend if it matches certain criteria::

sub vcl_fetch {

if (beresp.url ~ “.(png|gif|jpg)$”) {

unset beresp.http.set-cookie;

set beresp.ttl = 3600;

}

}

#Example 3 - ACLs¶

#You create a named access control list with the acl keyword. You can match the IP address of the client against an ACL with the match operator.:

Who is allowed to purge….

acl local {

“localhost”;

“10.0.0.0.0”/24; / and everyone on the local network /

! “10.0.0.0.23”; / except for the dialin router /

}

sub vcl_recv {

if (req.request == “PURGE”) {

if (client.ip ~ local) {

return(lookup);

}

}

}

sub vcl_hit {

if (req.request == “PURGE”) {

set obj.ttl = 0s;

error 200 “Purged.”;

}

}

sub vcl_miss {

if (req.request == “PURGE”) {

error 404 “Not in cache.”;

}

}
补充几条相关命令
查看Varnish状态
/usr/local/varnish/bin/varnishstat -n /var/vcache/

查看访问最多的Referer

/usr/local/varnish/bin/varnishtop -n /var/vcache/ -i rxheader -I Referer

查看访问最多的URL

/usr/local/varnish/bin/varnishtop -n /var/vcache/ -i rxurl

官方文档:
http://www.varnish-cache.org/docs/2.1/

FAQ:
1、安装出错
configure: error: Package requirements (libpcre) were not met:
No package ‘libpcre’ found
答:yum install pcre-devel -y

文章目录
  1. 1. Error “} obj.status “ “ obj.response {“
    1. 1.0.1. Guru Meditation:
  • 2. This script run at 00:00
  • 3. Who is allowed to purge….
  • ,