Redis의 리샤딩

MOVE SLOT

클러스터는 이미 생성되어 있는 것을 가정합니다.

1. CLUSTER NODES

• cluster nodes

fdc57eb9234d4de8308d917a41d94fae879c1cea 192.168.100.101:6379@16379 myself,master - 0 1566187702000 1 connected 0-5460
c14494729aa5351a026ceffa02e3fcea36c3816d 192.168.100.102:6379@16379 master - 0 1566187705432 3 connected 10923-16383
258346726c0aeb52a05d70507e9fc3609087ee56 192.168.100.103:6379@16379 master - 0 1566187704430 2 connected 5461-10922

• new nodes

# 192.168.100.104
492b427590f391fc4b784a261239d09a0ab4ea94 :6379@16379 myself,master - 0 0 0 connected

2. 더미 데이터의 생성

# {test} tag에 의해 6918 슬롯으로 데이터가 저장됨
(for i in {0..100000}; do 
    redis-cli -p 6379 -h 192.168.100.102 -a password1 -c set {test}:${i} ${i} & 
done) >/dev/null 2>&1

3. 신규 노드 추가

redis-cli -h 192.168.100.104 -c CLUSTER MEET 192.168.100.101 6379

redis-cli -h 192.168.100.104 -c CLUSTER NODES
fdc57eb9234d4de8308d917a41d94fae879c1cea 192.168.100.101:6379@16379 myself,master - 0 1566187702000 1 connected 0-5460
c14494729aa5351a026ceffa02e3fcea36c3816d 192.168.100.102:6379@16379 master - 0 1566187705432 3 connected 10923-16383
258346726c0aeb52a05d70507e9fc3609087ee56 192.168.100.103:6379@16379 master - 0 1566187704430 2 connected 5461-10922
492b427590f391fc4b784a261239d09a0ab4ea94 192.168.100.104:6379@16379 myself,master - 0 1566277827000 0 connected

4. 이동할 슬롯의 MIGRATING/IMPORTING 상태 설정

반드시, 타겟 슬롯의 상태부터 IMPORTING으로 변경할 것!
ref: https://redis.io/commands/cluster-setslot#redis-cluster-live-resharding-explained

# 192.168.100.104:6379 (TARGET)
CLUSTER SETSLOT 6918 IMPORTING 492b427590f391fc4b784a261239d09a0ab4ea94
# 192.168.100.102:6379 (SOURCE)
CLUSTER SETSLOT 6918 MIGRATING 258346726c0aeb52a05d70507e9fc3609087ee56

5. 데이터 마이그레이션

# CLUSTER GETKEYSINSLOT slot의 출력결과가 없을 때까지 반복
CLUSTER GETKEYSINSLOT 6918 3
1) "{test}:0"
2) "{test}:1"
3) "{test}:10"
 
MIGRATE 192.168.100.104 6379 "" 0 1000 AUTH password1 KEYS {test}:0 {test}:1 {test}:10

• 참고: redis-trib.rb에서는 한 번에 10개씩 키를 옮기며, timeout는 60초이다.

6. SLOT 할당에 대한 선언

# 전체 클러스터 노드에서 실행해줄 필요가 있음
redis-cli -p 6379 -h 192.168.100.101 -c CLUSTER SETSLOT 6918 492b427590f391fc4b784a261239d09a0ab4ea94
redis-cli -p 6379 -h 192.168.100.102 -c CLUSTER SETSLOT 6918 492b427590f391fc4b784a261239d09a0ab4ea94
redis-cli -p 6379 -h 192.168.100.103 -c CLUSTER SETSLOT 6918 492b427590f391fc4b784a261239d09a0ab4ea94
redis-cli -p 6379 -h 192.168.100.104 -c CLUSTER SETSLOT 6918 492b427590f391fc4b784a261239d09a0ab4ea94

7. ROLLBACK

1. MIGRATING/IMPORTING이 계속 적용되어 있는 상태라면, 마저 마이그레이션을 진행한다.

# IMPORTING 노드로 옮겨진 키가 있는 상태에서, MIGRATING에 아직 존재하는 동일한 키가 있을 수 있으므로
# REPLACE 옵션이 추가되어야 한다.
# ref: https://redis.io/commands/migrate
MIGRATE __TARGET_IP__ __TARGET_PORT__ "" 0 1000 REPLACE AUTH password1 KEYS keys

2. IMPORTING만 적용되어 있는 상태라면, 현재 slot의 주인 노드로 데이터를 옮긴다.

MIGRATE __SOURCE_IP__ __SOURCE_PORT__ "" 0 1000 AUTH password1 KEYS keys
# 마이그레이션 완료 이후, IMPORTING만 해제하는 것이므로, STABLE 커맨드로 간단히 ROLLBACK한다.
CLUSTER SETSLOT 6918 STABLE

3. MIGRATING만 적용되어 있는 상태이고, 해당 슬롯에 데이터가 없다면 그대로 STABLE을 실행한다.

CLUSTER GETKEYSINSLOT 6918 10
CLUSTER SETSLOT 6918 STABLE

• 참고: redis-trib.rb (v3.0)의 fix_open_slots의 일부 코드

# Case 1: The slot is in migrating state in one slot, and in
#         importing state in 1 slot. That's trivial to address.
if migrating.length == 1 && importing.length == 1
    move_slot(migrating[0],importing[0],slot,:dots=>true,:fix=>true)
# Case 2: There are multiple nodes that claim the slot as importing,
# they probably got keys about the slot after a restart so opened
# the slot. In this case we just move all the keys to the owner
# according to the configuration.
elsif migrating.length == 0 && importing.length > 0
    xputs ">>> Moving all the #{slot} slot keys to its owner #{owner}"
    importing.each {|node|
        next if node == owner
        move_slot(node,owner,slot,:dots=>true,:fix=>true,:cold=>true)
        xputs ">>> Setting #{slot} as STABLE in #{node}"
        node.r.cluster("setslot",slot,"stable")
    }
# Case 3: There are no slots claiming to be in importing state, but
# there is a migrating node that actually don't have any key. We
# can just close the slot, probably a reshard interrupted in the middle.
elsif importing.length == 0 && migrating.length == 1 &&
      migrating[0].r.cluster("getkeysinslot",slot,10).length == 0
    migrating[0].r.cluster("setslot",slot,"stable")