技术

基于mysql+redis+iscsi+lvs+nfs+keepalived+nginx搭建高可用集群

By admin
1000047204

一、环境准备

关闭防火墙和selinux
systemctl stop firewalld
systemctl disable firewalld
sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config
setenforce 0
mkdir -p /etc/yum.repos.d/backup
mv /etc/yum.repos.d/C*  /etc/yum.repos.d/backup/ 
mount /dev/sr0 /mnt
cat > /etc/yum.repos.d/aa.repo << EOF
[centos]
name=本地镜像
baseurl=file:///mnt
enabled=1
gpgcheck=0 
EOF

二、MySQL数据库主从复制

（一）软件
yum install -y mariadb*  net-tools vim
（二）数据库：
master1配置
vim /etc/my.cnf

在[mysqld]下添加
log-bin=mysql-bin   #日志文件名称mysql-bin.00001
server-id=1    #0--65535范围
log_slave_updates=1 #开启级联
binlog-ignore-db=mysql
binlog-ignore-db=information_schema
binlog-ignore-db=performance_schema
binlog-ignore-db=sys
#replicate-do-db=master1   #同master二进制日志所监控的库名称相同
relay-log=mysql-relay-bin  #中继日志的名称
replicate-ignore-db=mysql
replicate-ignore-db=information_schema
replicate-ignore-db=performance_schema
replicate-ignore-db=sys	

master2配置
vim /etc/my.cnf

在[mysqld]下添加
log-bin=mysql-bin   #日志文件名称mysql-bin.00001
server-id=2   #0--65535范围
log_slave_updates=1 #开启级联
binlog-ignore-db=mysql
binlog-ignore-db=information_schema
binlog-ignore-db=performance_schema
binlog-ignore-db=sys
#replicate-do-db=master1   #同master二进制日志所监控的库名称相同
relay-log=mysql-relay-bin  #中继日志的名称
replicate-ignore-db=mysql
replicate-ignore-db=information_schema
replicate-ignore-db=performance_schema
replicate-ignore-db=sys

slave1配置
vim /etc/my.cnf

在[mysqld]下添加
server-id=3
#replicate-do-db=master1   #同master二进制日志所监控的库名称相
relay-log=mysql-relay-bin  #中继日志的名称
replicate-ignore-db=mysql
replicate-ignore-db=information_schema
replicate-ignore-db=performance_schema
replicate-ignore-db=sys	

slave2配置
vim /etc/my.cnf

在[mysqld]下添加
server-id=4
#replicate-do-db=master1   #同master二进制日志所监控的库名称相同
relay-log=mysql-relay-bin  #中继日志的名称
replicate-ignore-db=mysql
replicate-ignore-db=information_schema
replicate-ignore-db=performance_schema
replicate-ignore-db=sys

重启服务
systemctl restart mariadb
systemctl enable mariadb

master1、master2数据库授权
mysql -uroot –p000000
create user 'aa'@'10.10.0.%' identified by '000000';
grant replication slave on *.* to 'aa'@'10.10.0.%' identified by '000000';
flush privileges;
查看二进制日志
show master status;
master1

master2


配置slave1数据库连接到master1数据库
change master to master_host='10.10.0.90',master_user='aa', master_password='000000', master_port=3306, master_log_file='mysql-bin.000001',master_log_pos=313;
配置master2数据库连接到master1数据库
change master to master_host='10.10.0.90',master_user='aa', master_password='000000', master_port=3306, master_log_file='mysql-bin.000001',master_log_pos=313;

配置slave2数据库连接到master2数据库
change master to master_host='10.10.0.91',master_user='aa', master_password='000000', master_port=3306, master_log_file='mysql-bin.000002',master_log_pos=313;
配置master1数据库连接到master2数据库
change master to master_host='10.10.0.91',master_user='aa', master_password='000000', master_port=3306, master_log_file='mysql-bin.000002',master_log_pos=313;

开启复制
start slave;
查看slave状态
show slave status \G;


配置虚拟IP
vi  /etc/sysconfig/network-scripts/ifcfg-lo:0
添加
DEVICE=lo:0
IPADDR=10.10.0.110
NETMASK=255.255.255.255
ONBOOT=yes
重启网卡
systemctl restart network
添加路由
echo "route add -host 10.10.0.110 dev lo:0" >> /etc/rc.local
route add -host 10.10.0.110 dev lo:0
调整/proc内核参数，关闭ARP响应
vi   /etc/sysctl.conf
添加
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
保存退出
sysctl -p

三、Keepalived双机热备+LVS高可用群集

（一）软件：
yum -y install keepalived ipvsadm

（二）Keepalived
systemctl restart keepalived
systemctl enable keepalived
cd /etc/keepalived/
cp -p keepalived.conf keepalived.conf.bak
echo "" > keepalived.conf

Keepalived + lvs1配置
vi keepalived.conf
global_defs {  
   router_id 1
}
vrrp_instance VI_1 {   
    state MASTER
    interface ens33
    virtual_router_id 1
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {    
        10.10.0.110
    }
}
virtual_server 10.10.0.110  3306{
        delay_loop 15
        lb_algo rr
        lb_kind DR
        protocol TCP
        real_server 10.10.0.90 3306 {
        weight 1
        TCP_CHECK {
                connect_port 3306
                connect_timeout 3
                nb_get_retry 3
                delay_before_retry 4
                }
        }
        real_server 10.10.0.91 3306 {
        weight 1
        TCP_CHECK {
                connect_port 3306
                connect_timeout 3
                nb_get_retry 3
                delay_before_retry 4
                }
        }
}
保存退出	

Keepalived + lvs2配置

vi keepalived.conf
global_defs {  
   router_id 2
}
vrrp_instance VI_1 {   
    state BACKUP
    interface ens33
    virtual_router_id 1
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {    
        10.10.0.110
    }
}
virtual_server 10.10.0.110 3306 {
        delay_loop 15
        lb_algo rr
        lb_kind DR
        protocol TCP
        real_server 10.10.0.90 3306 {
        weight 1
        TCP_CHECK {
                connect_port 3306
                connect_timeout 3
                nb_get_retry 3
                delay_before_retry 4
                }
        }
        real_server 10.10.0.91 3306 {
        weight 1
        TCP_CHECK {
                connect_port 3306
                connect_timeout 3
                nb_get_retry 3
                delay_before_retry 4
                }
        }
}
保存退出

开启内核的服务模块
modprobe ip_vs

查看系统模块运行状态
lsmod | grep ip_vs

开机自动加载ip_vs模块
echo "modprobe ip_vs" >> /etc/rc.local
systemctl restart keepalived

四、Keepalived双机热备+Nginx高可用群集

（一）安装Nginx
配置yum源
cat >> /etc/yum.repos.d/aa.repo << EOF

[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/
gpgcheck=0
enabled=1
EOF

修改网关
vi /etc/resolv.conf

（二）软件
yum install -y nginx keepalived
启动并加入开机自启动
systemctl start nginx keepalived
systemctl enable nginx keepalived

（三）配置Nginx
修改/etc/nginx/nginx.conf
vi   /etc/nginx/nginx.conf
http {
	...
    upstream luntan {
        server 10.10.0.100:80;    
        server 10.10.0.101:80;     
    }
}
	
修改/etc/nginx/conf.d/default.conf
vi /etc/nginx/conf.d/default.conf
server {
	...
	location / {
        proxy_pass http://luntan;
        proxy_redirect default; 
        #设置代理 
        proxy_set_header Host $host;       
        proxy_set_header X-Real-IP $remote_addr; 
    }
}


（四）配置Keepalived双机热备
cd /etc/keepalived/
cp -p keepalived.conf keepalived.conf.bak
echo "" > keepalived.conf

keepalived 1配置
vi keepalived.conf

global_defs {  
   router_id 3
}
vrrp_instance VI_1 {   
    state MASTER
    interface ens33
    virtual_router_id 2
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {    
        10.10.0.120
    }
}
保存退出	

keepalived 2
vi keepalived.conf

global_defs {  
   router_id 4
}
vrrp_instance VI_1 {   
    state BACKUP
    interface ens33
    virtual_router_id 2
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {    
        10.10.0.120
    }
}
保存退出

（五）重启nginx和keepalived
systemctl restart nginx
systemctl restart keepalived

五、iSCSI配置

（一）软件
yum install -y mdadm targetcli 

（二）创建RAID（RAID1或RAID2）
mdadm -C /dev/md1 -l5 -n3 -x1 /dev/sd[b,c,d,e]

（三）创建LVM
vgcreate my_vg /dev/md1
lvcreate my_vg -n my_lv1 -L 20G
lvcreate my_vg -n my_lv2 -L 20G
lvcreate my_vg -n my_lv3 -L 20G

（四）配置target
targetcli
backstores/block/create disk1 /dev/my_vg/my_lv1

（五）创建target
iscsi/ create #系统会自动设置iqn名称
iscsi/ create iqn.2018-11.cn.edu.sziit:masterstore #如需手动设置iqn名称，参照iqn命名规范
#iSCSI target名称格式为：
#iqn.< 年份-月份 >.< 域名反写 > [: < 设备识别;可以是任意字符

（六）配置target-tpg1
1、创建LUN（逻辑单元号）
cd iscsi/iqn.2018-11.cn.edu.sziit:masterstore/
tpg1/luns create /backstores/block/disk1
tpg1/luns create /backstores/block/disk2

2、配置portal（门户）
tpg1/portals/ create 10.10.0.96:3260

3、配置acl（访问控制列表）
tpg1/acls create iqn.2018-11.cn.edu.sziit:web1

（七）配置target-tpg2 
1、创建tpg（Target Portal Group） 
cd iscsi/iqn.2018-11.cn.edu.sziit:masterstore/ 
create tpg2 

2、创建LUN（逻辑单元号） 
tpg2/luns create /backstores/block/disk3 

3、配置portal（门户） 
tpg2/portals/ create 10.10.0.96 3260

4、配置acl（访问控制列表） 
tpg2/acls create iqn.2018-11.cn.edu.sziit:fileserver1

配置 iSCISI 客户端
（一）安装iscsi-initiator-utils 
yum install -y iscsi-initiator-utils 

（二）配置initiatorname.iscsi 
vi /etc/iscsi/initiatorname.iscsi 
写入：
InitiatorName=iqn.2018-11.cn.edu.sziit:web1 

（三）配置服务 
systemctl enable iscsi iscsid 
systemctl restart iscsi iscsid 

（三）发现 
man iscsiadm 
iscsiadm --mode discoverydb --type sendtargets --portal 10.10.0.96:3260 --discover

（四）登录 
iscsiadm --mode node --targetname iqn.2018-11.cn.edu.sziit:masterstore --portal 10.10.0.96:3260 -- login 

iSSCISI 客户端使用虚拟磁盘 
（一）分区 
fdisk  //如整块磁盘仅分一个分区，此步可省略 
（二）格式化 
mkfs.xfs /dev/sdx 
mkfs.ext4 /dev/sdy 
（三）自动挂载 
mkdir /NetDisk/{disk1,disk2} -p 
blkid 
echo 'UUID="4d76c8f5-3bfb-4bcd-9785-a0b78faaaf62" /NetDisk/disk1 xfs defaults,_netdev 0 0' >> /etc/fstab  
//UUID不会发生变化，/dev/sdx可能发生变化 
echo'UUID="704f81ab-7148-45c6-ba4d-630ad96968c6"/NetDisk/disk2 ext4 defaults,_netdev 0 
0' >> /etc/fstab 
mount -a 
df -h

六、NFS配置

（一）软件
yum install -y nfs-utils* mdadm iscsi-initiator-utils

（二）配置
挂载iSCSI
配置initiatorname.iscsi 
vi /etc/iscsi/initiatorname.iscsi 
InitiatorName=iqn.2018-11.cn.edu.sziit:web1 

配置服务 
systemctl enable iscsi iscsid 
systemctl restart iscsi iscsid 

发现 
man iscsiadm 
iscsiadm --mode discoverydb --type sendtargets --portal 10.10.0.96:3260 --discover 


登录 
iscsiadm --mode node --targetname iqn.2018-11.cn.edu.sziit:masterstore --portal 10.10.0.96:3260 --login 

创建共享目录
mkdir -p /luntan

创建RAID（安装mdadm）
mdadm -C /dev/md1 -l5 -n2 -x1 /dev/sd[b,c,d]
mkfs.ext4 /dev/md1
mount /dev/md1 /luntan
mdadm -Ds /dev/md1 >> /etc/mdadm.conf

配置NFS
vi /etc/exports

添加
/luntan *(rw,no_all_squash)  #共享目录 可连接的IP （权限）
启动
systemctl restart rpcbind nfs
systemctl enable rpcbind nfs

重读NFS文件
exportfs -r

（三）上传文件
解压文件
授权

七、web服务器

（一）软件
yum install -y httpd* php* nfs* wget

（二）挂载NFS（mount NFS服务器IP:/共享目录 挂载点）
mount 10.10.0.97:/luntan /var/www/html

启动httpd
systemctl restart httpd

（三）配置虚拟IP
vi  /etc/sysconfig/network-scripts/ifcfg-lo:0
添加
DEVICE=lo:0
IPADDR=120
NETMASK=255.255.255.255
ONBOOT=yes
重启网卡
systemctl restart network
添加路由
echo "route add -host 10.10.0.120 dev lo:0" >> /etc/rc.local
route add -host 10.10.0.120 dev lo:0

调整/proc内核参数，关闭ARP响应
vi   /etc/sysctl.conf
添加
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
保存退出
sysctl -p
此图片的 alt 属性为空；文件名为 %E5%B1%8F%E5%B9%95%E6%88%AA%E5%9B%BE-2024-10-14-105617.png

八、keepalived双机热备+redis

（一）Redis配置（两台机子一致）
yum install -y wget keepalived
wget  http://download.redis.io/releases/redis-2.8.13.tar.gz 
tar -zxf redis-2.8.13.tar.gz
cd redis-2.8.13
make PREFIX=/usr/local/redis  install
cp  redis.conf  /usr/local/redis/

启动redis
nohup  /usr/local/redis/bin/redis-server  /usr/local/redis/redis.conf  &

关闭redis
/usr/local/redis/bin/redis-cli  -p  6379 shutdown

（二）keepalived热备
systemctl start keepalived
cd /etc/keepalived/
cp -p keepalived.conf keepalived.conf.bak
echo "" > keepalived.conf

keepalived 1配置
vi keepalived.conf

global_defs {  
   router_id 5
}
vrrp_instance VI_1 {   
    state MASTER
    interface ens33
    virtual_router_id 3
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {    
        10.10.0.130
    }
}
保存退出	

keepalived 2 配置
vi keepalived.conf

global_defs {  
   router_id 6
}
vrrp_instance VI_1 {   
    state BACKUP
    interface ens33
    virtual_router_id 3
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {    
        10.10.0.130
    }
}
保存退出

重启keepalvied
systemctl restart keepalived
systemctl enable keepalived

（三）安装PHP-Redis连接驱动（web主机上）
wget  https://github.com/phpredis/phpredis/archive/3.1.2.tar.gz 
tar xzf  3.1.2.tar.gz
cd phpredis-3.1.2/
phpize    注：如果失败需要安装环境php-devel,如果继续失败请确认版本是否符合
./configure  --with-php-config=/usr/bin/php-config  --enable-redis
make && make install

在/etc/php.ini中添加redis.so模块 
vim /etc/php.ini
extension=redis.so

验证
vi   /var/www/html/aa.php
<?php
phpinfo();
?>
通过浏览器访问，http://10.10.0.120/aa.php，查看是否有redis模块。


测试
打开全局配置文件config_global.php.
vi  /var/www/html/config/config_global.php
查找CONFIG MEMORY段，将redis server后改为10.10.0.130
此图片的 alt 属性为空；文件名为 image-7-1024x760.png

By admin

Related Post

技术

Heat模板使用

admin
技术

RabbitMQ服务优化

admin
技术

OpenStack秘钥管理器服务

admin

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注