Redis三主三从

准备：两台机器 192.168.1.121 192.168.1.122 搭建三主三从的集群服务

首先在 192.168.1.121 机器上操作：在 usr/local/redis-cluster 目录下新建 7001、7002、7003文件夹。

1.在 usr/local/ 目录下新建 redis-cluster文件夹

在 usr/local/redis-cluster 目录下执行安装ruby的命令：yum install ruby （谨慎使用）

出现如下界面：

然后输入y 回车

建议提前安装好ruby 2.xx 以上的版本。默认安装的2.0一下的版本，redis集群不支持。详情见笔记30 ruby的安装

然后在该目录下继续执行命令：

yum install rubygems

gem install  redis

如果输入：gem install redis 报错： Could not find a valid gem 'redis' (>= 0), here is why:

解决：输入命令： gem sources 然后再输入： gem install redis

Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources

步骤二：

把redis.conf 配置文件拷贝到7001下，修改文件内容，如下：

daemonize yes //后台启动 port 7001 dir "/usr/local/redis-cluster/7001" //（指定文件保持的位置，必须要保存在不同的文件夹下） appendonly yes //开启主从复制 bind 192.168.1.121 //(必须要绑定当前机器的ip) cluster-enable yes //开启集群模式 cluster-config-file nodes-7001.conf //保存了其他节点，redis的状态，持久变量等等。 cluster-node-timeout 5000 //如果主节点通过5秒还没有反应，通过类似投票机制，其他节点顶上。

参考效果：

命令模式下搜索:/cluster

步骤三：启动集群：

/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7001/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7002/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7003/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7004/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7005/redis.conf
/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7006/redis.conf

查看是否启动成功：ps -el|grep redis

步骤四：创建集群

到redis-3.0.2安装目录下，执行redis-trib.rb 命令

cd /usr/local/redis-3.0.2/src/ 
./redis-trib.rb  create --replicas 1 192.168.1.121:7001 192.168.1.121:7002 192.168.1.121:7003 192.168.1.122:7004 192.168.1.122:7005 192.168.1.122:7006

备注： --replicas 1 后面跟着所有的节点的ip和端口，其中1 代表每个主节点配置几个从节点。

replicas 复制， --replicas 1 为集群中的每个主节点都创建一个从节点。

出现yes然后出现以下信息

步骤五：验证集群，进入到bin目录下，连接当前机器的节点

cd /usr/local/redis/bin/
./redis-cli -c -h 192.168.1.121 -p 7001

cluster info  查看集群信息 cluser nodes 查看节点信息 192.168.1.121:7001> cluster nodes 5c7524cab01eaed054f7677dacc48ea3e69fbf31 192.168.1.122:7005 slave 067868fcab2276185e144fec94da005f1f90f973 0 1545523255768 5 connected eb5d75d0e5f86faafd474013b3d8a27bff791b65 192.168.1.121:7003 slave da36a8cb2d6dd6910e2f64ca17117638918b232e 0 1545523254249 4 connected 067868fcab2276185e144fec94da005f1f90f973 192.168.1.121:7001 myself,master - 0 0 1 connected 0-5460 da36a8cb2d6dd6910e2f64ca17117638918b232e 192.168.1.122:7004 master - 0 1545523255264 4 connected 5461-10922 f18a18e0d35eb52f72b9db8cd284d3bfd1677b19 192.168.1.122:7006 slave 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 0 1545523255265 6 connected 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 192.168.1.121:7002 master - 0 1545523256268 2 connected 10923-16383

3.验证数据

192.168.1.121:7001> set name yangyang -> Redirected to slot [5798] located at 192.168.1.122:7004 OK 192.168.1.122:7004> get name "yangyang"

关闭集群（谨慎关闭，如果我们还要水平扩展集群，增加主，从节点的话，之前的服务都需要开着）

关闭集群需要逐个关闭每台集群

a. 如果当前机器还在redis客户端连接下，需要退出当前redis客户端连接

192.168.1.122:7004> quit

b. 使用关闭集群的命令

[root@localhost /]# /usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7001 shutdown [root@localhost /]# /usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7002 shutdown [root@localhost /]# /usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7003 shutdown [root@localhost /]# /usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7004 shutdown [root@localhost /]# /usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7005 shutdown [root@localhost /]# /usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7006 shutdown

以上是三主三从的集群服务搭建

Q: 水平扩展，在集群中如何增加新的主节点

A:现在需要新增一个主节点和一个从节点。master 7007 slave 7008具体步骤如下：

1 进入该目录下: cd /usr/local/redis-cluster/

2 在redis-cluster目录下新增7007文件夹，修改redis.conf配置文件，修改参照搭建集群时的步骤二。把 redis.conf文件夹上传到7007文件夹下。先增加的节点需要先开启集群操作（神坑，一定要做）

-因为我们为新节点分配槽的时候，需要知道新节点7007的ID

- 如果不开启新节点的服务，分配槽的时候不成功。

操作如下：

[root@localhost src]# /usr/local/redis/bin/redis-server /usr/local/redis-cluster/7007/redis.conf [root@localhost src]# /usr/local/redis/bin/redis-server /usr/local/redis-cluster/7008/redis.conf

此时我们进入进入任意一台redis-cli客户端下使用：cluster nodes 命令，可以查看新增加的两台机器70007 和 7008 未分配槽。

进入reids-3.0.2文件下，使用增加节点的命令add-node，其中192.168.1.121:7007 为新增的主节点，

192.168.1.121:7001为已经存在的任意节点。执行命令如下：

[root@localhost src]# cd /usr/local/redis-3.0.2/src [root@localhost src]# ./redis-trib.rb add-node 192.168.1.121:7007 192.168.1.121:7001

查看集群的状态：

192.168.1.121:7001> cluster nodes da36a8cb2d6dd6910e2f64ca17117638918b232e 192.168.1.122:7004 slave eb5d75d0e5f86faafd474013b3d8a27bff791b65 0 1545839185059 16 connected 067868fcab2276185e144fec94da005f1f90f973 192.168.1.121:7001 myself,slave 5c7524cab01eaed054f7677dacc48ea3e69fbf31 0 0 1 connected f18a18e0d35eb52f72b9db8cd284d3bfd1677b19 192.168.1.122:7006 slave 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 0 1545839183545 17 connected 73b17a0b74db74bb28fd9421ce13163838dba2c0 192.168.1.121:7007 master - 0 1545839183039 0 connected 5c7524cab01eaed054f7677dacc48ea3e69fbf31 192.168.1.122:7005 master - 0 1545839184050 13 connected 0-5460 eb5d75d0e5f86faafd474013b3d8a27bff791b65 192.168.1.121:7003 master - 0 1545839184050 16 connected 5798-10922 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 192.168.1.121:7002 master - 0 1545839184556 17 connected 5461-5797 10923-16383

4.为主节点分配槽

因为会用到新增节点的id，分配槽之前先执行以下命令：

启动新增加的redis服务7007，然后使用redis-trib.rb 中的check 命令。用来查看新增加的节点ID 是多少。

命令：

./redis-trib.rb  check 192.168.1.121:7007

[root@localhost src]# /usr/local/redis/bin/redis-server /usr/local/redis-cluster/7007/redis.conf [root@localhost src]# ps -ef|grep redis root      4682     1  1 08:36 ?        00:00:34 /usr/local/redis/bin/redis-server 192.168.1.121:7001 [cluster]             root      4708     1  0 08:45 ?        00:00:17 /usr/local/redis/bin/redis-server 192.168.1.121:7002 [cluster]             root      4713     1  0 08:45 ?        00:00:17 /usr/local/redis/bin/redis-server 192.168.1.121:7003 [cluster]             root      4767  4461  0 09:00 pts/0    00:00:00 ruby ./redis-trib.rb reshard 192.168.1.121:7001 root      4816     1  2 09:16 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.121:7007 [cluster]             root      4820  4662  0 09:16 pts/1    00:00:00 grep redis [root@localhost src]# ./redis-trib.rb  check 192.168.1.121:7007 Connecting to node 192.168.1.121:7007: OK >>> Performing Cluster Check (using node 192.168.1.121:7007) M: e788cb38cd1956b60e7c2f72549d7f2854ce170d 192.168.1.121:7007   slots: (0 slots) master   0 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [ERR] Not all 16384 slots are covered by nodes.

执行如下分配槽的命令：

[root@localhost src]# ./redis-trib.rb reshard 192.168.1.121:7001

提示后输入槽大小1-16384：

How many slots do you want to move (from 1 to 16384)? 4000 What is the receiving node ID? 73b17a0b74db74bb28fd9421ce13163838dba2c0 Please enter all the source node IDs.  Type 'all' to use all the nodes as source nodes for the hash slots.  Type 'done' once you entered all the source nodes IDs. Source node #1: all Do you want to proceed with the proposed reshard plan (yes/no)? yes 执行完误报：[ERR] ERR Syntax error, try CLIENT (LIST | KILL ip:port | GETNAME | SETNAME connection-name)

再来查看集群节点信息，发现7007 已经从其他节点分配了槽。

192.168.1.121:7001> cluster nodes da36a8cb2d6dd6910e2f64ca17117638918b232e 192.168.1.122:7004 slave eb5d75d0e5f86faafd474013b3d8a27bff791b65 0 1545839702631 16 connected 067868fcab2276185e144fec94da005f1f90f973 192.168.1.121:7001 myself,slave 5c7524cab01eaed054f7677dacc48ea3e69fbf31 0 0 1 connected f18a18e0d35eb52f72b9db8cd284d3bfd1677b19 192.168.1.122:7006 slave 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 0 1545839702123 17 connected 73b17a0b74db74bb28fd9421ce13163838dba2c0 192.168.1.121:7007 master - 0 1545839700607 18 connected 0-1332 5461-5797 10923-12001 5c7524cab01eaed054f7677dacc48ea3e69fbf31 192.168.1.122:7005 master - 0 1545839701616 13 connected 1333-5460 eb5d75d0e5f86faafd474013b3d8a27bff791b65 192.168.1.121:7003 master - 0 1545839702122 16 connected 5798-10922 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 192.168.1.121:7002 master - 0 1545839700607 17 connected 12002-16383

以上增加集群主节点操作已经完成。

总结：主要用到命令 ./redis-trib.rb add-node 192.168.1.121:7007 192.168.1.121:7001 ./redis-trib.rb reshared 192.168.1.121:7001

Q: 如何增加集群中的一个从节点

1.先启动7008这台服务

/usr/local/redis/bin/redis-server /usr/local/redis-cluster/7008/redis.conf

   2.  把7008加到集群中

./redis-trib.rb add-node 192.168.1.122:7008 192.168.1.121:7001

3.进入到7008连接的客户端

/usr/local/redis/bin/redis-cli -c -h 192.168.1.122 -p 7008

给7008子节点指定一个主节点

cluster replicate 主节点ID

查看是否添加成功，发现7008已变为子节点，并且分配了槽

192.168.1.121:7001> cluster nodes da36a8cb2d6dd6910e2f64ca17117638918b232e 192.168.1.122:7004 slave eb5d75d0e5f86faafd474013b3d8a27bff791b65 0 1545844631369 16 connected 067868fcab2276185e144fec94da005f1f90f973 192.168.1.121:7001 myself,master - 0 0 19 connected 1333-5460 f18a18e0d35eb52f72b9db8cd284d3bfd1677b19 192.168.1.122:7006 slave 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 0 1545844631874 17 connected 529b1515f46b4818dc711c0a6b56751ca8afa20a 192.168.1.122:7008 slave 73b17a0b74db74bb28fd9421ce13163838dba2c0 0 1545844631773 18 connected 73b17a0b74db74bb28fd9421ce13163838dba2c0 192.168.1.121:7007 master - 0 1545844629856 18 connected 0-1332 5461-5797 10923-12001 5c7524cab01eaed054f7677dacc48ea3e69fbf31 192.168.1.122:7005 slave 067868fcab2276185e144fec94da005f1f90f973 0 1545844629857 19 connected eb5d75d0e5f86faafd474013b3d8a27bff791b65 192.168.1.121:7003 master - 0 1545844630360 16 connected 5798-10922 65e1dcda6aceadcb27bd0fb6f76626a5a040ba78 192.168.1.121:7002 master - 0 1545844630864 17 connected 12002-16383

end

Q: 如何删除集群中的一个已经存在的从节点

./redis-trib.rb  del-node 192.168.1.7007   ID(从节点id)

Q: 如何删除集群中的一个已经存在的主节点

主节点分配了槽，需要把删除的主节点上的槽分配到其他主节点上。进行重新分片操作，使用如下命令：

./redis-trib.rb  reshard  192.168.1.121:7007

./redis-trib.rb  del-node  192.168.1.121:7007  ID(要删除节点的id)

redis.conf

在某移动客服系统实际使用配置

daemonize yes
pidfile /home/zhangpeng/redis/redis1/log/redis-7001.pid
port 7001
bind 172.19.195.40
cluster-enabled yes
cluster-config-file nodes-7001.conf
dir /home/zhangpeng/redis/redis1/data
cluster-node-timeout 2000
cluster-require-full-coverage no
aof-rewrite-incremental-fsync yes
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel verbose
logfile "/home/zhangpeng/redis/redis1/log/redis-7001.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error no
rdbcompression yes
rdbchecksum yes
dbfilename dump-7001.rdb
dir /home/zhangpeng/redis/redis1/data
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly yes
appendfilename "appendonly-7001.aof"
dir /home/zhangpeng/redis/redis1/data
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
maxclients 15000

Redis哨兵

uu898.com项目中实际使用

参考文档

https://www.jianshu.com/p/f76300000eae

https://www.jianshu.com/p/729266ecabe7

https://www.cnblogs.com/linuxbug/p/5131504.html

https://blog.csdn.net/pengjunlee/article/details/81429119

https://blog.csdn.net/shouhuzhezhishen/article/details/69221517

1、启动所有nodes

2、连接所有nodes，在命令行执行：slaveof 192.168.0.143 21025

3、连接nodes，检测是否加入集群，启动哨兵

config/21025.conf
bind 0.0.0.0
protected-mode yes
port 21025
daemonize yes
tcp-backlog 511
timeout 0
tcp-keepalive 300
supervised no
pidfile /home/uu898/sentinel-redis/log/21025.pid
logfile "/home/uu898/sentinel-redis/log/21025.log"
dir /home/uu898/sentinel-redis/data/
loglevel notice
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbchecksum yes
dbfilename dump_21025.rdb
masterauth 111111
requirepass 111111
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
maxclients 10000
lazyfree-lazy-eviction no
lazyfree-lazy-expire n
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly no
appendfilename "appendonly-21025.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
config/21028.conf
protected-mode no
port 21028
daemonize yes
dir /home/uu898/sentinel-redis/data
pidfile /home/uu898/sentinel-redis/data/sentinel_21028.pid
logfile /home/uu898/sentinel-redis/log/sentinel_21028.log
sentinel monitor mymaster 10.10.90.132 21025 2
sentinel auth-pass mymaster 111111
sentinel down-after-milliseconds mymaster 1000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 3000
sentinel deny-scripts-reconfig yes

Keys	Action
`?`	Open this help
`n`	Next page
`p`	Previous page
`s`	Search