Redis Getting Started: From Key to Data Types
Remote Dictionary Server
Redis Ten Major Data Types
Key
Commands are case insensitive Keys are case sensitive
View all keys
keys *exists keytype keydel keyunlink keyNon blocking deletion, it only removes the keys fromkeyspacethe metadata, and the actual deletion happens asynchronously later.del keyIt is an atomic deletion, and only when the deletion succeeds will the deletion result be returned. If you are deleting a large key, using del will block all subsequent operations, while unlink key does not block, it deletes data asynchronously in the backgroundttl keyView how many seconds until expiration, -1 means never expires, -2 means already expiredexpire keySet an expiration time for the given keymove key dbindex[0-15]Move the key of the current database to the given database DBselect dbindexSwitch database (0-15), default is 0dbsizeflushdb [ASYNC/SYNC]flushallhelp @type,help @hash
keys pattern keys * keys set*
exists key
type key
del key

String String
string is redis the most basic type,one key corresponds to one value
string the type is binary safe which means redis of string it can contain any data, such as jpg images or serialized objects.
string the type is Redis the most basic data type, one redis the string in value can be at most 512M
List (list)
Redis A list is a simple list of strings, sorted by insertion order. You can add an element to the head (left side) or tail (right side) of the list
Its underlying implementation is actually a doubly linked list, and it can contain at most 2^32-1 elements (4294967295, each list can exceed 4 billion elements)
A simple list of strings, sorted by insertion order

lpush key value1 value2 value3lrange key start stop0 represents the head, -1 represents the tailrpop keyllen keylindex key indexllen keylrem key number valuenumber=0 represents allltrim key index1 index2Delete from indexindex1to indexindex2the value ofrpoplpush key1 key2Pop the last data item of key1 to key2
lset key index valueReplacelinsert before value1 value2/aftervalue1 value2before inserts value 2 before value 1
The last inserted one is in the first position
Hash (hash)
A String type mapping table of field and value, hash is especially suitable for storing objects
Redis hash is a string type mapping table of field and value, hash is especially suitable for storing objects.Redis each one in hash can store 2^32-1 key value pairs (more than 4 billion)

HSET key field valueHGET key fieldHDEL key fieldHKEYS keyHVALS keyhlen keyhexists keyhkeys/hvalshincrby/hincrbyfloathsetnx
Set (set)
single value multiple values, no duplicates, unordered and no duplicates, list is ordered and allows duplicates
String type unordered set, set members are unique, in the set cannot be repeated data of
Redis Set is an unordered collection of String type. Collection members are unique, which means duplicate data cannot appear in the collection. The encoding of the collection object can be intset or hashtable
In Redis, the Set collection is implemented through a hash table, so the complexity of adding, deleting, and searching is all O(1)
The maximum number of members in a collection is 2^32-1 (4294967295, each collection can store more than 4 billion members)
kepptllDefault after modificationttlwill become-1, need to specify unchanged
mset/mget batch
getrange/setrange

INCR
INCRBY
DECR
DECRBY
STRLEN k1 xxx (String Length)
APPEND key value

sadd key nember1 number2 number3Duplicates are automatically removed when addedsmembers keytraversescard keynumbersrem key member1 member2 member3deletesrandmember key [number]Randomly display from key[number]elements, elements are not deletedspop key[number]Randomly pop an element from the collection, pop one and delete onesdiff key1 key2:key 1-key 2result ofsinter key1 key2 key3intersectionsintercard numkeys key [key...] [LIMIT limit]also intersection, returns the count
sunion key1 key2 key3union
Only takes effect if all succeed
Sorted Set (ZSET)
String a collection of type elements, and duplicate members are not allowed, each element is associated with a double type score, sorted by score
zadd key score 1 name 1 score 2 name 2 score 3 name 3zscore key memberGet element scorezCARD keyGet the number of elements in the collectionzrangen key start stop [withscores]zincrby key increment memberzrem key member [memeber ...]deletezmpop number key min count 1zrank key values/zrevrank key valuesGet index value, reverse order or index value
Geospatial (GEO)
Garbled text: using Chinese connection reids when need to add --raw
Redis GEO Mainly used to store geographic location information and operate on the stored information
(longitude,latitude)
Add coordinates of the geographic location.
Get coordinates of the geographic location.
Calculate the distance between two locations.
Based on the longitude and latitude coordinates given by the user, get the ones within the specified rangelocation set
geoadd key longitude latitude nameGEOADD city 116.403963 39.915119 "天安门" 116.403414 39.924091 "故宫" 116.024067 40.362639 "长城"geohashReturn the coordinatesgeohashrepresentation generatebase32encoded value, representing the coordinates
geodist key name1 name2 [m|km|ft|mi]automatically calculate the distance between two places
georadiusUsing the given longitude and latitude as the center, return all position elements in the key whose distance from the center does not exceed the given maximum distance
redis Cardinality statistics (HyperLogLog)
Stores the count of data, does not store actual content
HyperLogLog is used for cardinality statistics algorithm, with deduplication statistics function, is String type
Advantage : when the number or volume of input elements is very very large, the space required to calculate the cardinality is always fixed and very small.
HyperLogLogis a probabilistic algorithm, what is stored is the count of data
In Redis, eachHyperLogLlog key only needs to spend12KBmemory, and can calculate close to2^64the cardinality of different elements. This forms a sharp contrast with sets, which consume more memory the more elements there are when calculating cardinality.
However, becauseHyperLogLogonly calculates the cardinality based on input elements, whiledoes not store the input elements themselves, soHyperLogLogcannot return each input element like a set does.
Count roughly how many people have visited a page, the same IP only represents one, deduplicate and count unique visitors
pfadd key value1 value2 value3pfcount keypfmerge key key1 key2Merge key1 , key 2 , does not store specific values, only tells you the count
One sentence: the real data after deduplication and extraction

redis Bitmap (bitmap)
a binary bit represented by 0 and 1 states bit array, equivalent to a subclass of string, the type is still string
Whether logged in before, whether clicked, clock in and out for work, sign in statistics
Use mysql the amount of data is too large
setbit key position valuevalue can only be 0 or 1getbit key positionstrlenCount how many bytes are occupied, 8 bits make one bitmapsetbit k1 1 1setbit k1 7 1occupies one bytesetbit k1 1 1setbit k1 8 1occupies two bytesbitcount keystatistics
redis Stream
Redis Stream is a data structure newly added in Redis 5.0 version.
Redis StreamMainly used for message queues(MQ,Message Queue),Redisis itself aRedispublish subscribe(pub/sub)to implement the message queue function, but it has a disadvantage that messages cannot be persisted. If network disconnection, Redis downtime, etc. occur, messages will be discarded.
Simply put, publish subscribe(pub/sub)can distribute messages, but cannot record historical messages.
WhileRedis Streamprovides message persistence and master slave replication functions, allowing any client to access data at any time, and can remember each client access position, and can also ensure messages are not lost.
queue related commands
Queue: namely the sequence produced IDx
queue related commands
xadd queue name * key1 value1 key2 value2message id must be larger than the previous one,*used to let the system automatically generate id, will generate message ID
xrange mstream - +
xdel key IDxlen key - +xtrim key maxlen numberthe maximum count, the earlier ones will be truncated
xread mstream - +
blocking returns only when there is the maximum, will tell you how long you waited
consumption related commands
XREADGROUP GROUP group name consumer name STREAMS queue name
Because the cursor has already gone from head to tail, even the same group cannot read
but different groups can read

xpending queue name group nameQuery all consumers within each consumer group
xpending queue name group name - + count consumerView which data a certain consumer has read
After acknowledging, querying the read data again will only leave the unacknowledged ones
redis Bitfield (understand)
Treat the redis string as an array composed of binary digits, and it can address and modify variable-length bit widths and any specified integer bitfield without byte alignment
Through the bitfield command, multiplebit fields(referring to multiple consecutive bits), it will execute a series of operations and return a response array, where the elements in this array correspond to the corresponding execution results in the parameter list.
Simply put, through the bitfield command we canoperate on multiple bit fields at onceperform operations.
ASCII code | ASCII code reference table
Bitfield modification, overflow control
Equivalent to being able to modify bytecode
-128 to 127, anything exceeding this overflows
Two colons will result in empty

redis Persistence
Persistence refers to the writing of data to durable storage
- Restore via RDB snapshot files
(dump.rab), restoring the disk snapshot to memory - Restore via commands recorded in AOF
RDB
Enable: save time num Takes effect when triggered num times within time
Disable: save "" or redis-cli config set save ""(non permanent) You can still manually save/bgsave generate RDB
Path modification
Filename modification: search the configuration file for
dump.Search the configuration file
config get dirGet the directoryProblem encountered: need to
dumpfilesChange folder permissions
SAVE Blocks, so other commands cannot be processed; prohibited for use in production
BGSAVE: fork a child process to perform replication persistence, while the main process can modify data simultaneously
fork()
lastsave date -d
RDB pros and cons
Repair
Optimize configuration
yes: if during thebgsaveprocess, a write error occurs,Rediswill stop accepting any requests untilbgsaveis completedno: even if during thebgsaveprocess a write error occurs,Redisit will continue to accept requests and attempt to write
rdbchecksum Safety and consistency, recommended yes

rdb-del-sync-files no need to worry about it
AOF
three write-back strategies
appendsync everysec
generally choose everysec, always will cause too much IO and performance degradation
no depends on linux the kernel data flush configuration, usually 30 seconds
save rules
repair redis-check-aof --fix filename
AOF rewrite mechanism
appendfsync every/always/no
auto- aof- rewrite- percentage 100set to 100 means that whenever the current AOF file size is twice or more the size of the previous AOF file
auto- aof- rewrite- min- size 1k
incr file reaches 1ktwice the size and it will rewrite
redis transaction
transaction requirement: hope that one command or multiple commands can be combined into an indivisible, atomic, uninterrupted, uniformly executed batch command
normal execution
multi
exec
abandon transaction
multi
discard
all suffer together
during inputan error is detected, redis will directly return an error and no command will be executed
each pays for their own debt
cannot detect in the early stageerrors that are only known at runtime, only the failed ones will not be executed

watch monitoring
Redis pipeline
Redis pipelining | Redis how to optimize the performance bottleneck caused by frequent command round trips
reduce RTT(Round Trip Time) the impact on performance
requires linux the client and the server to complete together

Redis publish and subscribe (understand)
is redis the predecessor prototype of the message queue

subscribe channel [channel ...]
Punsubscribe [pattern [pattern ...]]
redis replication (replica)
master-slave replication: master is mainly for writing, slave is mainly for reading when master data changes, automatically synchronize the new data asynchronously to other slave databases
configure replicas but not masters
configuration
---common part---
bind 0.0.0.0 # Allow address
protected-mode: yes #Prevent Redis The instance, without any security configuration such as requirepass/bind), is accessed by the outside world.
daemonize yes # Allow background startup
logfile "/usr/local/redis/redis.log" # Log path
dir /opt/software/redis-7.0.3/data # RDB File storage directory
appenddirname "appendonlydir" # AOF The file will be at dir at the location of appenddirname folder save
appendfilename "appendonly.aof" # AOF File name
appendonly yes # AOF Enable
`redis` Master slave: Read write separation, disaster recovery, data backup, horizontal scaling to support high concurrency
Master slave:slave can still serve as another slave of the master(Reduce master pressure of)
Only the topmost master can execute commands,slave can copy the previous one and can also copy
When the master goes down, the slave will wait for the master to come online
Change midway:The slave can change the master at any time, and after the change it will automatically clear the data between them and copy the new master of the
Drawbacks:
If it goes down, manual intervention is needed
When the system is busy,slave causes high latency
--- Master.conf-----
requirepass xxx #master Set
--- Master slave Slave.conf(Only the slave configuration needs to be changed, no need to change the master configuration)------
# Specify the master node IP and port(Recommended)
replicaof Master IP Master port
# Or use the old way
slaveof Master IP Master port
slaveof no one #Stop synchronization between the current database and other databases
masterauth xxx #slave Set
---- Command line ----
info replication #View information
slaveof Master IP Master port # Execute manually, valid for the current session
Replication principles and workflow
redis Sentinelsentinel
Unattended operations and maintenance : will slave select through some mechanism to perform master slave replacement
Function Master slave monitoring: monitor master and slave redis operation Message notification: send failover results to clients Failover: automatic master slave switching Configuration center: obtain the current Redis server master node address by connecting to sentinel
sentinel.conf Configuration file
Configure with the replication configured in the previous chapter
sentinel.conf(配置好一个后直接拷贝改动)
Configuration file
Explanation
sentinel monitor <master-name> <ip> <redis-port> <quorum>
quorum : 2 that is, 2 sentinels agreeing is enough
Network congestion, hacker attacks, network fluctuations and so on, the sentinels communicate with each other to judge whether there is really a failure, sometimes it is just a problem with a certain sentinel itself
sentinel auth-pass <master-name> <password>
![]()
Start the cluster
sentinel log
conf The file is modified automatically
Shut down master Approach 1s , the background carried out an election, and at the same time slave. conf the file will also be modified
The one elected as the master conf The file content is deleted automatically
Is elected as slave is automatically modified

The first failure is because the network needs to be re-read and re-planned, telling you that the original connection was disconnected, and you may need to resend a heartbeat packet to reconnect
The remaining twoslaveOne of them will be elected asmaster
Even if the previous master comes back, it will not become the current master anymore
Minor issue
The running process of sentinels and the election principle
SDownSubjective down(Subjectively Down)
- Election Raft algorithm

- Driven forward by the sentinels
failoverprocess and elect a new master
- Elect a new
master
Replication offset (replication offset) : due to network and other reasons, the twoslaveare not necessarily the same size 2. Generatemaster, the others becomemasterofslave
3. Set the old one that has gone offline masteras a slave node of the newly elected master

- Elect a new
- Driven forward by the sentinels
Usage suggestions
Although unattended failover is possible : but electing sentinels leader and generating a new master, at the very least it takes 5-10s, the data during this period will be lost
-
redis Cluster (cluster)
redis-cli -a password -p 6385 -c
-c carry out routing and forwarding
The previous sentinels are no longer needed
Role
Cluster algorithm - sharding - slots slot
Reads and writes are both on the same slot

The key value will use CRC16 algorithm to perform a checksum, then take the modulo of 16384 to get a slot number (such as 1234 number)
Hash slot mapping
Hash modulo partitioning
Consistent hashing algorithm partitioning
Consistent hashing algorithm partitioning : all hash values form a complete set, and logical control makes them connect end to end to form a closed space, all hash values fall on the ring, and server nodes also fall on the ring based on their hashes, then the value will find the nearest server node clockwise for storage
Advantages: fault tolerance, scalability
Disadvantages: when there are too few servers, data skew problems occur
Propose a consistent hashing solution, with the goal of minimizing the impact on the mapping relationship from clients to servers when the number of servers changes
The algorithm constructs a consistent hash ring
keyKey placement rules that fall to servers
Advantages
Disadvantages
Hash slot partitioning
Hash slot partitioning: In data and node a layer is added in between hash slot (16384), to manage the relationship between data and nodes, slot The granularity problem is solved,hash the mapping problem is solved,
redis maps roughly equal hash slots to different nodes respectively
hash valuemodulo 16384 the remainder determines which corresponding slot it falls into
HASH_SLOT= CRC16(key) mod 16384
Why redis The maximum number of slots in a cluster is 16384 of them
redis The cluster does not guarantee strong consistency; under certain conditions, a redis cluster may lose some write request commands received by the system

Cluster environment setup steps
Three masters and three slaves redis Cluster configuration
1. Create new 6 configuration directories mkdis -p /myredis/cluster
2. configure 6 configuration files
3. Build master slave relationships
It may not be allocated as written; the actual allocation prevails
The master and slave configuration files are exactly the same, as follows (all 6 configuration files in the video contain the following content)
5000ms = 5s
- Build master slave relationships
It may not be allocated as written; the actual allocation prevails
redis-cli -a password --cluster create --cluster-replicas x
ip1:port1 ip2:port2 ip3:port3 ip4:port4 ip5:port5 ip6:port6
Using 6381 as the entry point, view and verify the cluster relationships
cluster nodes
Three masters and three slaves redis Cluster read and write
Due to slot restrictions, it is no longer possible to store things however you want; routing to the correct location is required
Internally, because there is CRC16 and a redirection algorithm k1 redirected to another slot
Connecting to the cluster requires adding c, so that what is connected is the large redis
redis-cli -a password -p 6385 -c

cluster keyslot key
View the slot value of a certain key

Master slave fault tolerance switchover and migration cases
master After going down slave it will automatically take over, even if previously master it rejoins, it will also becomeslave
After recovery, it becomes a slave

Manual failover or node subordinate adjustment
CLUSTER NODES
After restoring the previous master, it still becomes master
Master slave scale out case
Add
redis-cli -a password --cluster add-node ip1:port1 ip:portip1:port1is the newly added addressip:portis the existing address After joining, no slot number will be assigned![]()
![]()
Check
After joining, no slot number will be assigned

Reassign slot numbers (
reshard)
redis-cli -a psw --cluster reshard ip:port
Fill in 16384/4=4096
Fill in the new

Slots have been assigned, currently no slave is attached

Master slave scale in case
redis-cli -a psw --cluster del-node ip:port ID
slave was successfully deleted, next the slots of 6387 will be cleared and reassigned (in this example the cluster gave them back to 6381)
redis-cli -a psw --cluster reshard ip:6381
Then enter 4096
Two are not needed slave , then afterwards 6387 Delete
redis-cli -a psw --cluster del-node ip:port ID
Common cluster operation commands and CRC16 Algorithm analysis
cluster keyslot key
cluster countkeysinslot 12707

Spring Use Redis
Redis of the Java There are many clients, several commonly used ones:
Jedis
Lettuce
Spring Data Redis
Spring Data Redis It is part of Spring, and highly encapsulates the Redis low level development package.
In Spring projects, you can use Spring Data Redis to simplify operations
RedisTemplate
When connecting, note :
auth is the subsequent authentication command of the client
password is the connection authentication configuration on the server side
yaml
server.port=7777
spring.application.name=redis7_study
# ========================logging=====================
logging.level.root=info
logging.level.com.atguigu.redis7=info
logging.pattern.console=%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger- %msg%n
logging.file.name=D:/Users/XCJY/Desktop/qianduan
logging.pattern.file=%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger- %msg%n
# ========================swagger=====================
spring.swagger2.enabled=true
#In springboot2.6.X combined with swagger2.9.X it will prompt documentationPluginsBootstrapper a null pointer exception,
#the reason is that in springboot2.6.X in it SpringMVC the default path matching strategy changed from AntPathMatcher changed to PathPatternParser,
# causing the error, the solution is to matching-strategy switch back to the previous ant_path_matcher
spring.mvc.pathmatch.matching-strategy=ant_path_matcher
# ========================redis standalone=====================
spring.redis.database=0
# modify to your real IP
spring.redis.host=192.168.111.185
spring.redis.port=6379
spring.redis.auth=111111
spring.redis.lettuce.pool.max-active=8
spring.redis.lettuce.pool.max-wait=-1ms
spring.redis.lettuce.pool.max-idle=8
spring.redis.lettuce.pool.min-idle=0
lettuceConnectionFactory Suitable for high concurrency
RedisConfig
@Configuration
public class RedisConfig
{
/**
* redis serialization tool configuration class, you must enable the configuration below
* 127.0.0.1:6379> keys *
* 1) "ord:102" serialized
* 2) "\xac\xed\x00\x05t\x00\aord:102" wild, not serialized
* this.redisTemplate.opsForValue(); //provides operations for string all methods of the type
* this.redisTemplate.opsForList(); // provides operations for list all methods of the type
* this.redisTemplate.opsForSet(); //provides operations for set all methods of
* this.redisTemplate.opsForHash(); //provides operations for hash all methods of the table
* this.redisTemplate.opsForZSet(); //provides operations for zset all methods of
* @param lettuceConnectionFactory
* @return
*/
@Bean
public RedisTemplate<String, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory)
{
//Start creating redis the template object...
RedisTemplate<String,Object> redisTemplate = new RedisTemplate<>();
//Set redis the connection factory object of
redisTemplate.setConnectionFactory(lettuceConnectionFactory);
//Settings key serialization method string
redisTemplate.setKeySerializer(new StringRedisSerializer());
//Settings value serialization method json,Use GenericJackson2JsonRedisSerializer replace default serialization
redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());
//set hash serialization method
redisTemplate.setHashKeySerializer(new StringRedisSerializer());
redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
//RedisTemplate after all properties are set,perform some initialization operations
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
}
SwaggerConfig
@Configuration
@EnableSwagger2
public class SwaggerConfig
{
@Value("${spring.swagger2.enabled}")
private Boolean enabled;
@Bean
public Docket createRestApi() {
return new Docket(DocumentationType.SWAGGER_2)
.apiInfo(apiInfo())
.enable(enabled)
.select()
.apis(RequestHandlerSelectors.basePackage("com.atguigu.redis7")) //your own package
.paths(PathSelectors.any())
.build();
}
public ApiInfo apiInfo() {
return new ApiInfoBuilder()
.title("springboot利用swagger2构建api接口文档 "+"\t"+ DateTimeFormatter.ofPattern("yyyy-MM-dd").format(LocalDateTime.now()))
.description("springboot+redis整合,有问题给管理员阳哥邮件:zzyybs@126.com")
.version("1.0")
.termsOfServiceUrl("https://www.atguigu.com/")
.build();
}
}
RedisTemplate operation class
Spring provided Redis operation class for data structure
ValueOperationsclass, providesRedis String APIoperateListOperationsclass, providesRedis List APIoperateSetOperationsclass, providesRedis Set APIoperateZSetOperationsclass, providesRedis ZSet(Sorted Set) APIoperateGeoOperationsclass, providesRedis Geo APIoperate
serialization issue
//ThreadLocalRandom.current(): get the thread local random number generator bound to the current thread
//.nextInt(1000): generate a 0 to 999 random integer between
//+1: add the generated random integer 1,make the range become 1 to 1000
int keyId= ThreadLocalRandom.current().nextInt(1000)+1;
String serialNO= UUID.randomUUID().toString();
Set redisTemplate used by default``
want linux client to view Chinese --raw
otherwise it is garbled
![]()
reason
the default is JDK serialization method, which causes garbled text, by manually specifying to replace the default
connect cluster
connect cluster configuration
spring.redis.password=111111
# failed to get maximum redirection count
spring.redis.cluster.max-redirects=3
spring.redis.lettuce.pool.max-active=8
spring.redis.lettuce.pool.max-wait=-1ms
spring.redis.lettuce.pool.max-idle=8
spring.redis.lettuce.pool.min-idle=0
#support cluster topology dynamic sensing refresh,adaptive topology refresh whether to use all available updates, default false close,not add master after hanging spring do not know
spring.redis.lettuce.cluster.refresh.adaptive=true
#scheduled refresh
spring.redis.lettuce.cluster.refresh.period=2000
spring.redis.cluster.nodes=192.168.111.175:6381,192.168.111.175:6382,192.168.111.172:6383,192.168.111.172:6384,192.168.111.174:6385,192.168.111.174:6386
#support cluster topology dynamic sensing refresh,adaptive topology refresh whether to use all available updates, default false close,not add master after hanging spring do not know
spring.redis.lettuce.cluster.refresh.adaptive=true
#scheduled refresh
spring.redis.lettuce.cluster.refresh.period=2000
materials
- getting started mindset: official website
- troubleshooting mindset: logs





















































































































































































