How to safely traverse Redis keys in production environments? Use the SCAN command. SCAN is a cursor iterative command of Redis, which traverses the key in incremental manner to avoid blocking the main thread. 1. Call the loop until the cursor is 0; 2. Set the COUNT parameter reasonably, default 10, and the amount of big data can be appropriately increased; 3. Filter specific mode keys in combination with MATCH; 4. Pay attention to the possible repeated return of keys, inability to ensure consistency, performance overhead and other issues; 5. Can be run during off-peak periods or processed asynchronously. For example: SCAN 0 MATCH user:* COUNT 100.

In the production environment, if you need to traverse the Redis key, it is not advisable to use the KEYS command directly because it will block the Redis main thread and affect service performance. The SCAN command is a safe alternative, which traverses the key incrementally without causing long blockages.

What is SCAN?

SCAN is a cursor iterative command provided by Redis to gradually traverse key collections in the database. Unlike KEYS , it returns only a portion of the results at a time and records the current progress through a cursor. The basic usage is as follows:

 SCAN cursor [MATCH pattern] [COUNT count]

• cursor : start cursor, first call is 0.
• MATCH : Optional parameter to filter keys that match a specific pattern.
• COUNT : It is recommended to return the quantity, the default is 10, but the actual return quantity is not necessarily equal to the value.

How to properly traverse keys using SCAN?

To use SCAN safely, you need to understand its execution process and some key considerations:

• Loop until the cursor is 0 : a new cursor is returned for each call, until the returned cursor is 0 indicates that the traversal is completed.
• Don't assume that the data remains unchanged : the keys in Redis may be modified, deleted or added, so SCAN does not guarantee a completely consistent result.
• Reasonably set COUNT parameters : Usually the default value is enough, but it can be appropriately increased when the amount of data is large (such as 100~1000) to reduce the number of network round trips.
• Use in combination with MATCH : If you only want to scan for keys of certain prefixes or patterns, you can narrow the scope and improve efficiency through MATCH .

For example:

 127.0.0.1:6379> SCAN 0 MATCH user:* COUNT 100

How to implement SCAN in code?

Different language clients have slightly different encapsulation of SCAN , but the general logic is the same. Take Python's redis-py library as an example:

 import redis

client = redis.StrictRedis(host='localhost', port=6379, db=0)
cursor = 0
While True:
    cursor, keys = client.scan(cursor, match="user:*", count=100)
    for key in keys:
        # Process each key, such as deletion, viewing, etc. print(key)
    if cursor == 0:
        break

What should be noted is:

• Each call to scan() returns a new cursor and a batch of keys.
• Exit the loop when the cursor is 0.
• If the processing logic is heavy, you can add a queue to the queue for asynchronous processing after getting a batch of keys each time.

Limitations and Coping Methods of SCAN

While SCAN is safe, it also has some limitations:

• Possible to return key repeatedly : Due to Redis dictionary expansion and other reasons, a key may appear in multiple batches.
• Unable to guarantee consistency : When production environment key changes frequently, the data returned by SCAN may be an incomplete view of "snapshot".
• Performance overhead remains : while not blocking the main thread, frequent calls may still increase CPU and memory pressure.

To deal with these problems, you can:

• Run scan tasks during off-peak periods;
• For key operations (such as batch deletion), tests are performed first before going online;
• Combined with Lua scripts or batch processing, ensure logical idempotence and avoid repeated operation errors.

Basically that's it. The key to using SCAN is to understand that it is incremental and non-blocking, and also to note that it is not omnipotent.

The above is the detailed content of How to safely iterate over keys in production using the SCAN command?. For more information, please follow other related articles on the PHP Chinese website!