Remote Dictionary Server

assets/Redis入门/image-20231017214847414.png

Redis Ten Major Data Types

Key

Commands are case insensitive Keys are case sensitive

  • View all keys keys *

  • exists key

  • type key

  • del key

  • unlink key Non blocking deletion, it only removes the keys from keyspace the metadata, and the actual deletion happens asynchronously later. del key It is an atomic deletion, and only when the deletion succeeds will the deletion result be returned. If you are deleting a large key, using del will block all subsequent operations, while unlink key does not block, it deletes data asynchronously in the background

  • ttl keyView how many seconds until expiration, -1 means never expires, -2 means already expired

  • expire key Set an expiration time for the given key

  • move key dbindex[0-15] Move the key of the current database to the given database DB

  • select dbindex Switch database (0-15), default is 0

  • dbsize

  • flushdb [ASYNC/SYNC]

  • flushall

  • help @type , help @hash

keys pattern keys * keys set* exists key type key del key ../../../实战项目/assets/苍穹外卖/image-20231105225952599.png

String String

string is redis the most basic type,one key corresponds to one value

string the type is binary safe which means redis of string it can contain any data, such as jpg images or serialized objects.

string the type is Redis the most basic data type, one redis the string in value can be at most 512M

  • SET key value
  • Get key
  • SETEX key seconds value
  • SETNX key value Set if not exists ../../../实战项目/assets/苍穹外卖/image-20231105202010994.png

List (list)

../../../实战项目/assets/苍穹外卖/image-20231105205301485.png assets/Redis入门/image-20231121205027427.png

Redis A list is a simple list of strings, sorted by insertion order. You can add an element to the head (left side) or tail (right side) of the list

Its underlying implementation is actually a doubly linked list, and it can contain at most 2^32-1 elements (4294967295, each list can exceed 4 billion elements)

A simple list of strings, sorted by insertion order assets/Redis入门/image-20231121205430305.png

  • lpush key value1 value2 value3
  • lrange key start stop 0 represents the head, -1 represents the tail
  • rpop key
  • llen key
  • lindex key index
  • llen key
  • lrem key number value number=0 represents all
  • ltrim key index1 index2 Delete from index index1 to index index2 the value of
  • rpoplpush key1 key2 Pop the last data item of key1 to key2 assets/Redis入门/image-20231121210221569.png
  • lset key index value Replace
  • linsert before value1 value2/aftervalue1 value2 before inserts value 2 before value 1 assets/Redis入门/image-20231121210705633.png

assets/Redis入门/image-20231121205648659.png assets/Redis入门/image-20231121205704258.png

../../../实战项目/assets/苍穹外卖/image-20231105205257829.png

../../../../实战项目/assets/苍穹外卖/image-20231105204947520.png ../../../实战项目/assets/苍穹外卖/image-20231105205019432.png The last inserted one is in the first position

Hash (hash)

../../../实战项目/assets/苍穹外卖/image-20231105205328325.png

A String type mapping table of field and value, hash is especially suitable for storing objects

Redis hash is a string type mapping table of field and value, hash is especially suitable for storing objects.
Redis each one in hash can store 2^32-1 key value pairs (more than 4 billion) assets/Redis入门/image-20231121214424602.png

  • HSET key field value
  • HGET key field
  • HDEL key field
  • HKEYS key
  • HVALS key
  • hlen key
  • hexists key
  • hkeys/hvals
  • hincrby/hincrbyfloat
  • hsetnx

../../../../实战项目/assets/苍穹外卖/image-20231105205322936.png

assets/Redis入门/image-20231121215510513.png

Set (set)

single value multiple values, no duplicates, unordered and no duplicates, list is ordered and allows duplicates

../../../实战项目/assets/苍穹外卖/image-20231105210036482.png assets/Redis入门/image-20231121215858544.png

String type unordered set, set members are unique, in the set cannot be repeated data of

Redis Set is an unordered collection of String type. Collection members are unique, which means duplicate data cannot appear in the collection. The encoding of the collection object can be intset or hashtable In Redis, the Set collection is implemented through a hash table, so the complexity of adding, deleting, and searching is all O(1) The maximum number of members in a collection is 2^32-1 (4294967295, each collection can store more than 4 billion members)

assets/Redis入门/image-20231121200638105.png assets/Redis入门/image-20231121200839424.png

  • kepptll Default after modification ttl will become-1, need to specify unchanged

mset/mget batch getrange/setrange assets/Redis入门/image-20231121201854839.png assets/Redis入门/image-20231121201952299.png

INCR INCRBY DECR DECRBY

assets/Redis入门/image-20231121202052446.png STRLEN k1 xxx (String Length) APPEND key value assets/Redis入门/image-20231121202136946.png

  • sadd key nember1 number2 number3 Duplicates are automatically removed when added

  • smembers key traverse

  • scard key number

  • srem key member1 member2 member3 delete

  • srandmember key [number] Randomly display from key[number]elements, elements are not deleted

  • spop key[number] Randomly pop an element from the collection, pop one and delete one

  • smove key1 key2 value Migrate the value in key1 to key2 assets/Redis入门/image-20231121220621472.png

  • sdiff key1 key2 : key 1-key 2result of

  • sinter key1 key2 key3 intersection

  • sintercard numkeys key [key...] [LIMIT limit] also intersection, returns the count assets/Redis入门/image-20231121220933551.png

  • sunion key1 key2 key3 union

../../../../实战项目/assets/苍穹外卖/image-20231105210029219.png

assets/Redis入门/image-20231121201629268.png Only takes effect if all succeed

Sorted Set (ZSET)

../../../../实战项目/assets/苍穹外卖/image-20231105210503319.png

String a collection of type elements, and duplicate members are not allowed, each element is associated with a double type score, sorted by score

  • zadd key score 1 name 1 score 2 name 2 score 3 name 3
  • zscore key member Get element score
  • zCARD key Get the number of elements in the collection
  • zrangen key start stop [withscores]
  • zincrby key increment member
  • zrem key member [memeber ...] delete
  • zmpop number key min count 1
  • zrank key values/zrevrank key values Get index value, reverse order or index value

../../../Project/assets/苍穹外卖/image-20231105210408970.png

assets/Redis入门/image-20231122192919337.png

Geospatial (GEO)

Garbled text: using Chinese connection reids when need to add --raw

Redis GEO Mainly used to store geographic location information and operate on the stored information (longitude,latitude)

Add coordinates of the geographic location. Get coordinates of the geographic location. Calculate the distance between two locations. Based on the longitude and latitude coordinates given by the user, get the ones within the specified rangelocation set

assets/Redis入门/image-20231122204143924.png

  • geoadd key longitude latitude name GEOADD city 116.403963 39.915119 "天安门" 116.403414 39.924091 "故宫" 116.024067 40.362639 "长城"

  • geopos Return longitude and latitude assets/Redis入门/image-20240228120420573.png

  • geohash Return the coordinates geohash representation generate base32 encoded value, representing the coordinates assets/Redis入门/image-20240228120448132.png

  • geodist key name1 name2 [m|km|ft|mi] automatically calculate the distance between two places assets/Redis入门/image-20231122210330694.png

  • georadius Using the given longitude and latitude as the center, return all position elements in the key whose distance from the center does not exceed the given maximum distance assets/Redis入门/image-20231122210524494.png

  • georadiusbymember assets/Redis入门/image-20231122205359002.png

redis Cardinality statistics (HyperLogLog)

Stores the count of data, does not store actual content

HyperLogLog is used for cardinality statistics algorithm, with deduplication statistics function, is String type

Advantage : when the number or volume of input elements is very very large, the space required to calculate the cardinality is always fixed and very small.

HyperLogLogis a probabilistic algorithm, what is stored is the count of data

In Redis, eachHyperLogLlog key only needs to spend12KBmemory, and can calculate close to2^64the cardinality of different elements. This forms a sharp contrast with sets, which consume more memory the more elements there are when calculating cardinality.

However, becauseHyperLogLogonly calculates the cardinality based on input elements, whiledoes not store the input elements themselves, soHyperLogLogcannot return each input element like a set does.

Count roughly how many people have visited a page, the same IP only represents one, deduplicate and count unique visitors

assets/Redis入门/image-20231122202717716.png assets/Redis入门/image-20231122202857682.png

  • pfadd key value1 value2 value3
  • pfcount key
  • pfmerge key key1 key2 Merge key1 , key 2 , does not store specific values, only tells you the count

One sentence: the real data after deduplication and extraction assets/Redis入门/image-20240227201541008.png

assets/Redis入门/image-20231122203556920.png

redis Bitmap (bitmap)

a binary bit represented by 0 and 1 states bit array, equivalent to a subclass of string, the type is still string

assets/Redis入门/image-20231121190255065.png Whether logged in before, whether clicked, clock in and out for work, sign in statistics Use mysql the amount of data is too large

  • setbit key position value value can only be 0 or 1
  • getbit key position
  • strlen Count how many bytes are occupied, 8 bits make one bitmap setbit k1 1 1 setbit k1 7 1 occupies one byte setbit k1 1 1 setbit k1 8 1 occupies two bytes
  • bitcount key statistics

assets/Redis入门/image-20231122200304768.png

assets/Redis入门/image-20231122200255908.png

assets/Redis入门/image-20231122200827153.png

redis Stream

Redis Stream is a data structure newly added in Redis 5.0 version.

Redis StreamMainly used for message queues(MQ,Message Queue),Redisis itself aRedispublish subscribe(pub/sub)to implement the message queue function, but it has a disadvantage that messages cannot be persisted. If network disconnection, Redis downtime, etc. occur, messages will be discarded.

Simply put, publish subscribe(pub/sub)can distribute messages, but cannot record historical messages.

WhileRedis Streamprovides message persistence and master slave replication functions, allowing any client to access data at any time, and can remember each client access position, and can also ensure messages are not lost.

Before 5.0 assets/Redis入门/image-20231123092240065.png assets/Redis入门/image-20231123092226220.png assets/Redis入门/image-20231123092353082.png

assets/Redis入门/image-20231123092836878.png assets/Redis入门/image-20231123093213705.png

queue related commands

Queue: namely the sequence produced IDx

queue related commands

  • xadd queue name * key1 value1 key2 value2 message id must be larger than the previous one, * used to let the system automatically generate id, will generate message ID assets/Redis入门/image-20231123100537117.png

assets/Redis入门/image-20231123094601208.png assets/Redis入门/image-20231123094610916.png

  • xrange mstream - +

assets/Redis入门/image-20231123100544429.png assets/Redis入门/image-20231123100556622.png

  • xdel key ID

  • xlen key - +

  • xtrim key maxlen number the maximum count, the earlier ones will be truncated assets/Redis入门/image-20231123100609675.png

  • xread mstream - +

assets/Redis入门/image-20231123100707593.png assets/Redis入门/image-20231123100818606.png assets/Redis入门/image-20231123101137027.png assets/Redis入门/image-20231123101244474.png blocking returns only when there is the maximum, will tell you how long you waited

consumption related commands

assets/Redis入门/image-20231123112957755.png

  • xgroup queue name group name assets/Redis入门/image-20231123113100326.png assets/Redis入门/image-20231123113033042.png

  • XREADGROUP GROUP group name consumer name STREAMS queue name assets/Redis入门/image-20231123113313298.png Because the cursor has already gone from head to tail, even the same group cannot read but different groups can read assets/Redis入门/image-20231123113542833.png

assets/Redis入门/image-20231123114041008.png

  • xpending queue name group name Query all consumers within each consumer group

assets/Redis入门/image-20231123114405793.png

  • xpending queue name group name - + count consumer View which data a certain consumer has read assets/Redis入门/image-20231123114926474.png After acknowledging, querying the read data again will only leave the unacknowledged ones

xinfo stream queue name assets/Redis入门/image-20231123115317186.png

redis Bitfield (understand)

Treat the redis string as an array composed of binary digits, and it can address and modify variable-length bit widths and any specified integer bitfield without byte alignment

Through the bitfield command, multiplebit fields(referring to multiple consecutive bits), it will execute a series of operations and return a response array, where the elements in this array correspond to the corresponding execution results in the parameter list.

Simply put, through the bitfield command we canoperate on multiple bit fields at onceperform operations.

assets/Redis入门/image-20231123115707795.png assets/Redis入门/image-20231123115754768.png assets/Redis入门/image-20231123115826030.png ASCII code | ASCII code reference table

Bitfield modification, overflow control

assets/Redis入门/image-20231123120056197.png

Equivalent to being able to modify bytecode

assets/Redis入门/image-20231123120325015.png assets/Redis入门/image-20231123121343466.png

assets/Redis入门/image-20231123120639247.png

assets/Redis入门/image-20231123120652968.png -128 to 127, anything exceeding this overflows

assets/Redis入门/image-20231123121237132.png

../../../../实战项目/assets/苍穹外卖/image-20231115200200782.png ../../../实战项目/assets/苍穹外卖/image-20231115200205004.png Two colons will result in empty ../../../Project/assets/苍穹外卖/image-20231115200324146.png ../../../Project/assets/苍穹外卖/image-20231115200336930.png

redis Persistence

Redis persistence | Redis

Persistence refers to the writing of data to durable storage

  1. Restore via RDB snapshot files (dump.rab) , restoring the disk snapshot to memory
  2. Restore via commands recorded in AOF

RDB

Enable: save time num Takes effect when triggered num times within time Disable: save "" or redis-cli config set save ""(non permanent) You can still manually save/bgsave generate RDB

  • Path modification

  • Filename modification: search the configuration file for dump.

  • Search the configuration file config get dir Get the directory

  • Problem encountered: need to dumpfiles Change folder permissions

SAVE Blocks, so other commands cannot be processed; prohibited for use in production BGSAVE: fork a child process to perform replication persistence, while the main process can modify data simultaneously

assets/Redis入门/image-20231126151848905.png

fork()

fork() The function creates a child process completely identical to the parent process, including code, data, and open file descriptors, etc. assets/Redis入门/image-20231126151647572.png assets/Redis入门/image-20240411183509864.png

lastsave date -d

lastsaveUsed to get the timestamp of the most recent successful persistence operation (snapshot or AOF). It can tell you the time of the last completed persistence, which is very useful for monitoring and managing Redis instances assets/Redis入门/image-20240411183910192.png assets/Redis入门/image-20231126152035426.png date -d @16695455399 assets/Redis入门/image-20231126152053891.png

RDB pros and cons
  1. Suitable for large-scale data recovery
  2. Faster than AOF
  3. Scheduled backup

assets/Redis入门/image-20231126153008277.png assets/Redis入门/image-20231126153045751.png

  • RDB disadvantages
  1. Data is lost if an accident occurs before the save condition is triggered
  2. fork() may consume a lot of performance assets/Redis入门/image-20231126153320689.png assets/Redis入门/image-20231126153352477.png assets/Redis入门/image-20231126162523850.png
Repair

check -rdb /myredis/dumpfiles/dump6379.rdb

assets/Redis入门/image-20231126160209266.png A power outage midway through writing caused corruption assets/Redis入门/image-20231126160331485.png

assets/Redis入门/image-20231126160712998.png

Optimize configuration

assets/Redis入门/image-20231126161731124.png

  • stop-writes-on-bgsave-error assets/Redis入门/image-20231126161858568.png

  • yes: if during the bgsave process, a write error occurs, Redis will stop accepting any requests until bgsave is completed

  • no: even if during thebgsaveprocess a write error occurs,Redisit will continue to accept requests and attempt to write

assets/Redis入门/image-20231126162149866.png

  • rdbcopression becomes compressed and smaller, consuming a little cpu assets/Redis入门/image-20231126162225061.png

rdbchecksum Safety and consistency, recommended yes assets/Redis入门/image-20231126162254828.png

rdb-del-sync-files no need to worry about it

assets/Redis入门/image-20231126162436632.png

AOF

appendonly yes assets/Redis入门/image-20231126193414852.png

three write-back strategies

appendsync everysec

assets/Redis入门/image-20231126193529835.png generally choose everysec, always will cause too much IO and performance degradation

assets/Redis入门/image-20231126193756484.png no depends on linux the kernel data flush configuration, usually 30 seconds

save rules

assets/Redis入门/image-20231126195937792.png

incr file contents assets/Redis入门/image-20231126210856856.png

repair redis-check-aof --fix filename

if only data is truncated redis will not be affected,redis will automatically discard erroneous data, but if the erroneous data occurs in the middle of the file, the following data will also be discarded, possibly causing data loss

human tampering incr the file will fail to connect redis assets/Redis入门/image-20231129221629181.png

AOF rewrite mechanism

appendfsync every/always/no auto- aof- rewrite- percentage 100set to 100 means that whenever the current AOF file size is twice or more the size of the previous AOF file auto- aof- rewrite- min- size 1k incr file reaches 1ktwice the size and it will rewrite

assets/Redis入门/image-20231127115732684.png assets/Redis入门/image-20231127115842764.png

redis transaction

Transactions

transaction requirement: hope that one command or multiple commands can be combined into an indivisible, atomic, uninterrupted, uniformly executed batch command

assets/Redis入门/image-20231127155613382.png assets/Redis入门/image-20231127155650633.png

assets/Redis入门/image-20231127155803172.png assets/Redis入门/image-20231127155755041.png

normal execution

multi exec

abandon transaction

multi discard

all suffer together

during inputan error is detected, redis will directly return an error and no command will be executed

assets/Redis入门/image-20231127161053009.png

each pays for their own debt

cannot detect in the early stageerrors that are only known at runtime, only the failed ones will not be executed assets/Redis入门/image-20231127161546569.png assets/Redis入门/image-20231127161532114.png

watch monitoring

no queue jumping situation assets/Redis入门/image-20231127162554965.png

queue jumping situation assets/Redis入门/image-20231127162726154.png assets/Redis入门/image-20231127163027159.png

Redis pipeline

Redis pipelining | Redis how to optimize the performance bottleneck caused by frequent command round trips

assets/Redis入门/image-20231127190117404.png assets/Redis入门/image-20231127190122911.png assets/Redis入门/image-20231127190249430.png reduce RTT(Round Trip Time) the impact on performance

assets/Redis入门/image-20231127190300529.png

assets/Redis入门/image-20231127190801826.png

assets/Redis入门/image-20231127191453890.png requires linux the client and the server to complete together assets/Redis入门/image-20231127191824419.png

assets/Redis入门/image-20231127191846890.png

Redis publish and subscribe (understand)

assets/Redis入门/image-20231127192118410.png assets/Redis入门/image-20231127192133520.png

assets/Redis入门/image-20231127192035674.png is redis the predecessor prototype of the message queue assets/Redis入门/image-20231127192244259.png assets/Redis入门/image-20231127192255279.png assets/Redis入门/image-20231127192312758.png

assets/Redis入门/image-20231127192329346.png

  • subscribe channel [channel ...]

assets/Redis入门/image-20231127192355858.png

  • publish channel message assets/Redis入门/image-20231127192454696.png

  • publish pattern [pattern ...] assets/Redis入门/image-20231127192503194.png

  • pubsub subcommand [argument [argument ...]] assets/Redis入门/image-20231127192606406.png

  • unsubscribe [channel [channel ...]] assets/Redis入门/image-20231127192722621.png

  • Punsubscribe [pattern [pattern ...]]

assets/Redis入门/image-20231127192732320.png

redis replication (replica)

Redis replication | Redis

master-slave replication: master is mainly for writing, slave is mainly for reading when master data changes, automatically synchronize the new data asynchronously to other slave databases

assets/Redis入门/image-20231127194141359.png configure replicas but not masters

configuration


---common part---
bind 0.0.0.0 # Allow address
protected-mode: yes #Prevent Redis The instance, without any security configuration such as requirepass/bind), is accessed by the outside world.
daemonize yes               # Allow background startup
logfile "/usr/local/redis/redis.log"                # Log path
dir /opt/software/redis-7.0.3/data                 # RDB File storage directory
appenddirname "appendonlydir"  # AOF The file will be at dir at the location of appenddirname folder save
appendfilename "appendonly.aof" # AOF File name
appendonly yes                  # AOF Enable

`redis` Master slave: Read write separation, disaster recovery, data backup, horizontal scaling to support high concurrency

Master slave:slave can still serve as another slave of the master(Reduce master pressure of)
Only the topmost master can execute commands,slave can copy the previous one and can also copy 
When the master goes down, the slave will wait for the master to come online
Change midway:The slave can change the master at any time, and after the change it will automatically clear the data between them and copy the new master of the

Drawbacks: 
	If it goes down, manual intervention is needed
	When the system is busy,slave causes high latency
	
--- Master.conf-----
requirepass xxx #master Set

--- Master slave Slave.conf(Only the slave configuration needs to be changed, no need to change the master configuration)------
# Specify the master node IP and port(Recommended)
replicaof Master IP Master port
# Or use the old way
slaveof Master IP Master port
slaveof no one #Stop synchronization between the current database and other databases

masterauth xxx #slave Set

---- Command line ----

info replication #View information
slaveof Master IP Master port  # Execute manually, valid for the current session

Replication principles and workflow

assets/Redis入门/image-20231128082838348.png

redis Sentinelsentinel

assets/Redis入门/image-20231128083837925.png Unattended operations and maintenance : will slave select through some mechanism to perform master slave replacement

Function Master slave monitoring: monitor master and slave redis operation Message notification: send failover results to clients Failover: automatic master slave switching Configuration center: obtain the current Redis server master node address by connecting to sentinel

sentinel.conf Configuration file

Configure with the replication configured in the previous chapter

sentinel.conf(配置好一个后直接拷贝改动)


Configuration file

assets/Redis入门/image-20231202212105408.png assets/Redis入门/image-20231128162041506.png assets/Redis入门/image-20231128162104861.png assets/Redis入门/image-20231128162109484.png assets/Redis入门/image-20231128162114947.png assets/Redis入门/image-20231128162136182.png assets/Redis入门/image-20231128162129561.png

Explanation

assets/Redis入门/image-20231128162400248.png assets/Redis入门/image-20231128161705724.png sentinel monitor <master-name> <ip> <redis-port> <quorum> assets/Redis入门/image-20231128162455437.png quorum : 2 that is, 2 sentinels agreeing is enough assets/Redis入门/image-20231128161803719.png assets/Redis入门/image-20231128161809890.png Network congestion, hacker attacks, network fluctuations and so on, the sentinels communicate with each other to judge whether there is really a failure, sometimes it is just a problem with a certain sentinel itself sentinel auth-pass <master-name> <password> assets/Redis入门/image-20231128162738844.png

Start the cluster

assets/Redis入门/image-20231128163233329.png assets/Redis入门/image-20231128163700822.png

sentinel log

assets/Redis入门/image-20231128164037050.png assets/Redis入门/image-20231128164208041.png

conf The file is modified automatically

assets/Redis入门/image-20231128164435226.png

Shut down master Approach 1s , the background carried out an election, and at the same time slave. conf the file will also be modified

The one elected as the master conf The file content is deleted automatically

assets/Redis入门/image-20231128185117573.png

Is elected as slave is automatically modified assets/Redis入门/image-20231128184954431.png

assets/Redis入门/image-20231128185229310.png

assets/Redis入门/image-20231128164551422.png assets/Redis入门/image-20231128164737006.png

The first failure is because the network needs to be re-read and re-planned, telling you that the original connection was disconnected, and you may need to resend a heartbeat packet to reconnect

The remaining twoslaveOne of them will be elected asmaster

assets/Redis入门/image-20231128165424672.png

Even if the previous master comes back, it will not become the current master anymore

assets/Redis入门/image-20231128174912268.png

Minor issue

assets/Redis入门/image-20231128174728658.png

assets/Redis入门/image-20231128174756181.png

assets/Redis入门/image-20231128174813772.png

The running process of sentinels and the election principle

assets/Redis入门/image-20231128190252964.png

  1. SDown Subjective down (Subjectively Down)

assets/Redis入门/image-20231128190530847.png assets/Redis入门/image-20231128191904890.png

  1. ODown Objective down ( Objectively Down) assets/Redis入门/image-20231128192001394.png assets/Redis入门/image-20231128192020926.png
    1. Elect a sentinel

    assets/Redis入门/image-20231128192654194.png assets/Redis入门/image-20231128192617151.png assets/Redis入门/image-20231128192143544.png Elect one leader out of the three, and this leader elects the master 4. Election log assets/Redis入门/image-20231128192349190.png

  • Election Raft algorithm assets/Redis入门/image-20231128192506406.png
    1. Driven forward by the sentinels failover process and elect a new master assets/Redis入门/image-20231128193708167.png
      1. Elect a new master

      assets/Redis入门/image-20231128193445312.png assets/Redis入门/image-20231128192952244.png assets/Redis入门/image-20231128193141643.png Replication offset (replication offset) : due to network and other reasons, the two slave are not necessarily the same size 2. Generate master, the others become master of slave assets/Redis入门/image-20231128193431966.png 3. Set the old one that has gone offline master as a slave node of the newly elected master assets/Redis入门/image-20231128193459673.png assets/Redis入门/image-20231128193636342.png

Usage suggestions

assets/Redis入门/image-20231128193847788.png

Although unattended failover is possible : but electing sentinels leader and generating a new master, at the very least it takes 5-10s, the data during this period will be lost

-

redis Cluster (cluster)

redis-cli -a password -p 6385 -c -c carry out routing and forwarding

assets/Redis入门/image-20231202212649536.png assets/Redis入门/image-20231202212834197.png assets/Redis入门/image-20231202213448232.png assets/Redis入门/image-20231202213554725.png The previous sentinels are no longer needed

Role

assets/Redis入门/image-20231202213834193.png

Cluster algorithm - sharding - slots slot

assets/Redis入门/image-20231203131219819.png assets/Redis入门/image-20231203131116284.png

Advantages assets/Redis入门/image-20231203131040071.png

assets/Redis入门/image-20231203131230997.png Reads and writes are both on the same slot assets/Redis入门/image-20231203131554439.png

The key value will use CRC16 algorithm to perform a checksum, then take the modulo of 16384 to get a slot number (such as 1234 number)

assets/Redis入门/image-20231203132043842.png

Hash slot mapping

Hash modulo partitioning

assets/Redis入门/image-20231203144348748.png assets/Redis入门/image-20231203144408975.png

Consistent hashing algorithm partitioning

Consistent hashing algorithm partitioning : all hash values form a complete set, and logical control makes them connect end to end to form a closed space, all hash values fall on the ring, and server nodes also fall on the ring based on their hashes, then the value will find the nearest server node clockwise for storage

Advantages: fault tolerance, scalability

Disadvantages: when there are too few servers, data skew problems occur

assets/Redis入门/image-20231203235042370.png Propose a consistent hashing solution, with the goal of minimizing the impact on the mapping relationship from clients to servers when the number of servers changes

The algorithm constructs a consistent hash ring

assets/Redis入门/image-20231203235403225.png assets/Redis入门/image-20231203235412330.png assets/Redis入门/image-20231203235419050.png assets/Redis入门/image-20231204194821114.png

redis Server IP node mapping assets/Redis入门/image-20231204193546139.png assets/Redis入门/image-20231204193557277.png

key Key placement rules that fall to servers

assets/Redis入门/image-20231204194356866.png assets/Redis入门/image-20231204194413583.png

Advantages

  • Fault tolerance assets/Redis入门/image-20231204195936881.png assets/Redis入门/image-20231204195941607.png

  • Scalability

assets/Redis入门/image-20231204195956302.png assets/Redis入门/image-20231204200040523.png

Disadvantages

assets/Redis入门/image-20231204200101720.png assets/Redis入门/image-20231204200111578.png

Hash slot partitioning assets/Redis入门/image-20231204200408320.png assets/Redis入门/image-20231204200413940.png

Hash slot partitioning

Hash slot partitioning: In data and node a layer is added in between hash slot (16384), to manage the relationship between data and nodes, slot The granularity problem is solved,hash the mapping problem is solved,

redis maps roughly equal hash slots to different nodes respectively

hash valuemodulo 16384 the remainder determines which corresponding slot it falls into

assets/Redis入门/image-20231204200722768.png assets/Redis入门/image-20231204200821099.png assets/Redis入门/image-20231204200826047.png HASH_SLOT= CRC16(key) mod 16384

  • Hash value calculation assets/Redis入门/image-20231205083811562.png assets/Redis入门/image-20231205083818441.png assets/Redis入门/image-20231205083827192.png

Why redis The maximum number of slots in a cluster is 16384 of them

  1. assets/Redis入门/image-20231205084534724.png assets/Redis入门/image-20231205084554208.png assets/Redis入门/image-20231205084713411.png assets/Redis入门/image-20231205085258694.png assets/Redis入门/image-20231205085304306.png assets/Redis入门/image-20231205085318451.png

redis The cluster does not guarantee strong consistency; under certain conditions, a redis cluster may lose some write request commands received by the system assets/Redis入门/image-20231205085505374.png

Cluster environment setup steps

Three masters and three slaves redis Cluster configuration

assets/Redis入门/image-20231205230852024.png

1. Create new 6 configuration directories mkdis -p /myredis/cluster
2. configure 6 configuration files
3. Build master slave relationships

assets/Redis入门/image-20231205153658678.png assets/Redis入门/image-20231205153741767.png assets/Redis入门/image-20231205154742443.png It may not be allocated as written; the actual allocation prevails

assets/Redis入门/image-20231205153138122.png The master and slave configuration files are exactly the same, as follows (all 6 configuration files in the video contain the following content) assets/Redis入门/image-20231205152529952.png 5000ms = 5s

  1. Build master slave relationships

It may not be allocated as written; the actual allocation prevails

redis-cli -a password --cluster create --cluster-replicas x 
ip1:port1 ip2:port2 ip3:port3 ip4:port4 ip5:port5 ip6:port6

assets/Redis入门/image-20231205154211128.png assets/Redis入门/image-20231205154225253.png assets/Redis入门/image-20231205154317053.png assets/Redis入门/image-20231205154328501.png

Using 6381 as the entry point, view and verify the cluster relationships

info replication assets/Redis入门/image-20231205155031182.png assets/Redis入门/image-20231205155430747.png

cluster nodes

assets/Redis入门/image-20231205155115235.png assets/Redis入门/image-20231205155143405.png

Three masters and three slaves redis Cluster read and write

assets/Redis入门/image-20231205230916809.png assets/Redis入门/image-20231205161316784.png assets/Redis入门/image-20231205161350237.png Due to slot restrictions, it is no longer possible to store things however you want; routing to the correct location is required assets/Redis入门/image-20231205161521080.png assets/Redis入门/image-20231205161717414.png Internally, because there is CRC16 and a redirection algorithm k1 redirected to another slot

Connecting to the cluster requires adding c, so that what is connected is the large redis redis-cli -a password -p 6385 -c assets/Redis入门/image-20231205162004326.png

  • cluster keyslot key

View the slot value of a certain key assets/Redis入门/image-20231205162205983.png

Master slave fault tolerance switchover and migration cases

assets/Redis入门/image-20231205230907949.png

master After going down slave it will automatically take over, even if previously master it rejoins, it will also becomeslave assets/Redis入门/image-20231205165234327.png assets/Redis入门/image-20231205170302315.png After recovery, it becomes a slave assets/Redis入门/image-20231205170323087.png assets/Redis入门/image-20231205170339639.png

Manual failover or node subordinate adjustment

assets/Redis入门/image-20231205230829552.png

CLUSTER NODES

assets/Redis入门/image-20231205170556028.png After restoring the previous master, it still becomes master

Master slave scale out case

assets/Redis入门/image-20231205230817239.png

assets/Redis入门/image-20231205170824165.png

Add redis-cli -a password --cluster add-node ip1:port1 ip:port ip1:port1 is the newly added address ip:port is the existing address After joining, no slot number will be assigned assets/Redis入门/image-20231205173021176.png assets/Redis入门/image-20231205173035025.png assets/Redis入门/image-20231205173042672.png

Check

assets/Redis入门/image-20231205173056997.png After joining, no slot number will be assigned assets/Redis入门/image-20231205173102117.png

Reassign slot numbers (reshard)

redis-cli -a psw --cluster reshard ip:port

assets/Redis入门/image-20231205174241978.png Fill in 16384/4=4096 assets/Redis入门/image-20231205174344627.png Fill in the new assets/Redis入门/image-20231205174353465.png assets/Redis入门/image-20231205174409229.png assets/Redis入门/image-20231205174420847.png

Slots have been assigned, currently no slave is attached assets/Redis入门/image-20231205174526896.png assets/Redis入门/image-20231205175038245.png assets/Redis入门/image-20231205175226091.png

assets/Redis入门/image-20231205175404224.png assets/Redis入门/image-20231205175418201.png assets/Redis入门/image-20231205175431590.png

assets/Redis入门/image-20231205175548040.png

Master slave scale in case

assets/Redis入门/image-20231205230804953.png

assets/Redis入门/image-20231205230308788.png

redis-cli -a psw --cluster del-node ip:port ID assets/Redis入门/image-20231205231045083.png assets/Redis入门/image-20231205231253566.png slave was successfully deleted, next the slots of 6387 will be cleared and reassigned (in this example the cluster gave them back to 6381)

redis-cli -a psw --cluster reshard ip:6381 Then enter 4096

assets/Redis入门/image-20231205231545653.png assets/Redis入门/image-20231205231614693.png assets/Redis入门/image-20231205231816259.png Two are not needed slave , then afterwards 6387 Delete

redis-cli -a psw --cluster del-node ip:port ID

assets/Redis入门/image-20231205232000366.png

Common cluster operation commands and CRC16 Algorithm analysis

assets/Redis入门/image-20231205232531447.png

assets/Redis入门/image-20231205232249543.png assets/Redis入门/image-20231205232547770.png

assets/Redis入门/image-20231205232600766.png assets/Redis入门/image-20231205232616885.png assets/Redis入门/image-20231205232645850.png cluster keyslot key cluster countkeysinslot 12707 assets/Redis入门/image-20231205233046866.png

Spring Use Redis

Redis of the Java There are many clients, several commonly used ones: Jedis Lettuce Spring Data Redis

Spring Data Redis It is part of Spring, and highly encapsulates the Redis low level development package. In Spring projects, you can use Spring Data Redis to simplify operations

RedisTemplate

When connecting, note :

auth is the subsequent authentication command of the client password is the connection authentication configuration on the server side

yaml
server.port=7777

spring.application.name=redis7_study

# ========================logging=====================
logging.level.root=info
logging.level.com.atguigu.redis7=info
logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger- %msg%n 

logging.file.name=D:/Users/XCJY/Desktop/qianduan
logging.pattern.file=%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger- %msg%n

# ========================swagger=====================
spring.swagger2.enabled=true
#In springboot2.6.X combined with swagger2.9.X it will prompt documentationPluginsBootstrapper a null pointer exception,
#the reason is that in springboot2.6.X in it SpringMVC the default path matching strategy changed from AntPathMatcher changed to PathPatternParser,
# causing the error, the solution is to matching-strategy switch back to the previous ant_path_matcher
spring.mvc.pathmatch.matching-strategy=ant_path_matcher

# ========================redis standalone=====================
spring.redis.database=0
# modify to your real IP
spring.redis.host=192.168.111.185
spring.redis.port=6379
spring.redis.auth=111111
spring.redis.lettuce.pool.max-active=8
spring.redis.lettuce.pool.max-wait=-1ms
spring.redis.lettuce.pool.max-idle=8
spring.redis.lettuce.pool.min-idle=0

lettuceConnectionFactory Suitable for high concurrency

RedisConfig

@Configuration  
public class RedisConfig  
{  
    /**  
     * redis serialization tool configuration class, you must enable the configuration below  
     * 127.0.0.1:6379> keys *  
     * 1) "ord:102"  serialized  
     * 2) "\xac\xed\x00\x05t\x00\aord:102"   wild, not serialized  
     * this.redisTemplate.opsForValue(); //provides operations for string all methods of the type  
     * this.redisTemplate.opsForList(); // provides operations for list all methods of the type  
     * this.redisTemplate.opsForSet(); //provides operations for set all methods of  
     * this.redisTemplate.opsForHash(); //provides operations for hash all methods of the table  
     * this.redisTemplate.opsForZSet(); //provides operations for zset all methods of  
     * @param lettuceConnectionFactory  
     * @return  
     */  
    @Bean  
    public RedisTemplate<String, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory)  
    {  
        //Start creating redis the template object...  
        RedisTemplate<String,Object> redisTemplate = new RedisTemplate<>();  
        //Set redis the connection factory object of  
       redisTemplate.setConnectionFactory(lettuceConnectionFactory);  
        //Settings key serialization method string  
        redisTemplate.setKeySerializer(new StringRedisSerializer());  
        //Settings value serialization method json,Use GenericJackson2JsonRedisSerializer replace default serialization  
        redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());  
        //set hash serialization method  
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());  
        redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());  
		//RedisTemplate after all properties are set,perform some initialization operations
        redisTemplate.afterPropertiesSet();  
  
        return redisTemplate;  
    }  
}
SwaggerConfig
@Configuration  
@EnableSwagger2  
public class SwaggerConfig  
{  
    @Value("${spring.swagger2.enabled}")  
    private Boolean enabled;  
  
    @Bean  
    public Docket createRestApi() {  
        return new Docket(DocumentationType.SWAGGER_2)  
                .apiInfo(apiInfo())  
                .enable(enabled)  
                .select()  
                .apis(RequestHandlerSelectors.basePackage("com.atguigu.redis7")) //your own package  
                .paths(PathSelectors.any())  
                .build();  
    }  
    public ApiInfo apiInfo() {  
        return new ApiInfoBuilder()  
                .title("springboot利用swagger2构建api接口文档 "+"\t"+ DateTimeFormatter.ofPattern("yyyy-MM-dd").format(LocalDateTime.now()))  
                .description("springboot+redis整合,有问题给管理员阳哥邮件:zzyybs@126.com")  
                .version("1.0")  
                .termsOfServiceUrl("https://www.atguigu.com/")  
                .build();  
    }  
}

RedisTemplate operation class

Spring provided Redis operation class for data structure

  • ValueOperations class, provides Redis String API operate
  • ListOperations class, provides Redis List API operate
  • SetOperations class, provides Redis Set API operate
  • ZSetOperations class, provides Redis ZSet(Sorted Set) API operate
  • GeoOperations class, provides Redis Geo API operate

serialization issue

assets/Redis入门/image-20231206170034684.png

//ThreadLocalRandom.current(): get the thread local random number generator bound to the current thread  
//.nextInt(1000): generate a 0 to 999 random integer between  
//+1: add the generated random integer 1,make the range become 1 to 1000  
int keyId= ThreadLocalRandom.current().nextInt(1000)+1;  
String serialNO= UUID.randomUUID().toString();

Set redisTemplate used by default``

want linux client to view Chinese --raw otherwise it is garbled assets/Redis入门/image-20231206174534541.png assets/Redis入门/image-20231206171032961.png

reason

assets/Redis入门/image-20231206171347835.png assets/Redis入门/image-20231206171358790.png the default is JDK serialization method, which causes garbled text, by manually specifying to replace the default

assets/Redis入门/image-20231206171523544.png

connect cluster

connect cluster configuration
spring.redis.password=111111
# failed to get maximum redirection count
spring.redis.cluster.max-redirects=3
spring.redis.lettuce.pool.max-active=8
spring.redis.lettuce.pool.max-wait=-1ms
spring.redis.lettuce.pool.max-idle=8
spring.redis.lettuce.pool.min-idle=0

#support cluster topology dynamic sensing refresh,adaptive topology refresh whether to use all available updates, default false close,not add master after hanging spring do not know
spring.redis.lettuce.cluster.refresh.adaptive=true

#scheduled refresh
spring.redis.lettuce.cluster.refresh.period=2000
spring.redis.cluster.nodes=192.168.111.175:6381,192.168.111.175:6382,192.168.111.172:6383,192.168.111.172:6384,192.168.111.174:6385,192.168.111.174:6386

assets/Redis入门/image-20231207091438992.png assets/Redis入门/image-20231207091457707.png

solution assets/Redis入门/image-20231207091744697.png

https://github.com/lettuce-io/lettuce-core/wiki/Redis-Cluster#user-content-refreshing-the-cluster-topology-view

assets/Redis入门/image-20231207091636337.png assets/Redis入门/image-20231207091646755.png

#support cluster topology dynamic sensing refresh,adaptive topology refresh whether to use all available updates, default false close,not add master after hanging spring do not know
spring.redis.lettuce.cluster.refresh.adaptive=true

#scheduled refresh
spring.redis.lettuce.cluster.refresh.period=2000

materials

  1. getting started mindset: official website
  2. troubleshooting mindset: logs

Redis脑图(基础篇+高级篇) ↗