Duva - AOF/Save Operation Application Design

Duva

Duva is distributed in-memory database aimed at fast, efficient and scalable key-value store using Actor models written in Rust

Task

Building cache server, implement persistence feature with Save and AOF operation

Currently, Duva is primarily based on actor model - cache is not an exception.

Design

There are number of possible ways to implement save.
Perhaps, query manager supervises executions and get the result from each actors and save it. The problem of that approach is, however, subsequent queries may be blocked. Plus, it becomes extremely tricky when other set operation comes while in saving process.

Secondly, we can think of each Cache actor writes their own part to the same file as follows:

But still, it poses two challanges:

• Actors getting in race to get write access to file
• File header/metadata setting
• Cache actor taking responsibility of persistence
  • This is particularly problematic especially when we have to implement other features such as AOF

Therefore, here comes third, proposed approach:

In this design, Query manager(actor) spawns Save Actor when the save operation arrives, possibly with some other metadata such as number of cache actors.

And then save operation is sent to each cache actors so they can relay dataset they hold to Save actor which is responsible for persistence-related concern.

Now, let's say you want to have an option to turn on AOF done in background.

As persistence-related concern is isolated to Save actor, implementing it becomes quite easy:

With the AOF configration turned on, Save actor can optionally hold application buffer so you can further optimize by reducing IO call.