lvs+keepalived+nginx服务搭建过程

准备环境

服务 地址 环境
web服务器1 192.168.0.7:80 tomcat,nginx,realserver.sh
web服务器2 192.168.0.16:80 tomcat,nginx,realserver.sh
master 192.168.0.100 keepalived
backup 192.168.0.101 keepalived
  • VIP:192.168.0.80
  • 这里用的是keepalived的DR模式
  • 两台服务器都是centos7虚拟机

分别安装环境,ipvsadm,keepalived

1、安装环境依赖包,两台服务器都安装

1
# yum install -y gcc gcc-c++ make pcre pcre-devel kernel-devel openssl-devel libnl-devel popt-devel popt-static

2、安装 ipvsadm

1
2
3
4
# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
# tar zxf ipvsadm-1.26.tar.gz
# cd ipvsadm-1.26
# make ; make install

有系统有带的可以不安装
3、安装 keepalived

1
2
3
4
5
6
7
# tar xf keepalived-1.2.12.tar.gz
# cd keepalived-1.2.12
# ./configure --sysconf=/etc --sbindir=/usr/sbin/
# make && make install
# ln -s /usr/local/sbin/keepalived /sbin/keepalived
# chkconfig --add keepalived
# chkconfig --level 2345 keepalived on

  • 若/usr/src/kernels目录下没有内核目录,则需要安装内核开发包
    yum install -y kernel-devel kernel-headers安装
    yum install –y libnfnetlink-devel (libnfnetlink-devel-1.0.1-4.el7.x86_64.rpm)
  • CentOS7跟CentOS6的头文件路径有差别,把最后一个参数去掉
  • CentOS6下面编译
    ./configure –sysconf=/etc –with-kernel-dir=/usr/src/kernels/2.6.32-504.23.4.el6.x86_64

配置keepalived的主从文件

  • 192.168.0.100(MASTER)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
! Configuration File for keepalived

global_defs {
router_id MASTER
}


vrrp_instance VI_1 {
state MASTER
interface eno16777984
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.80
}
}

virtual_server 192.168.0.80 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 5
protocol TCP

real_server 192.168.0.16 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}

real_server 192.168.0.7 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
  • 192.168.0.101(BACKUP)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
! Configuration File for keepalived

global_defs {
router_id BACKUP
}
vrrp_instance VI_1 {
state BACKUP
interface ens160
virtual_router_id 51
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.80
}
}

virtual_server 192.168.0.80 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 5
protocol TCP

real_server 192.168.0.16 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}

real_server 192.168.0.7 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}

为07与16两台服务器添加VIP

脚本(realserver.sh)如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/bin/bash

VIP='192.168.0.80'

. /etc/init.d/functions

case "$1" in
start)
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce
echo " LVS Real-Server Start Success"
;;
stop)
/sbin/ifconfig lo:0 down
echo "0" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" > /proc/sys/net/ipv4/conf/all/arp_announce
echo " LVS Real-Server Stop Success"
;;
*)
echo "Usage: $0 ( start | stop )"
exit 1
esac

启动服务

1、启动16与07两台nginx

1
# service nginx start

2、分别在16与07执行脚本realserver.sh

1
# ./realserver.sh

启动后可以输入ifconfig看到

1
2
3
lo:0: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
inet 192.168.0.80 netmask 255.255.255.255
loop txqueuelen 1 (Local Loopback)

3、分别启动16与07的keepalived

1
# service keepalived start

1
2
3
4
5
6
7
[root@centoss1 home]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.80:80 rr persistent 5
-> 192.168.0.7:80 Route 1 0 0
-> 192.168.0.16:80 Route 1 0 0

4、访问服务
http://192.168.0.80

5、关掉两台服务器中的一个nginx
比如关掉16测试

分享