ARCANUM
Guide ยท saving

Saving rows

Say once which rows are saved and the game saves them by itself: some time after they change, when their player leaves, and when the server stops. This page goes from one line of setup to your own database.

Why saving is a declaration

A game server loses everything when the process ends: the gold a player earned, the chest that was opened. The usual answer is code that writes to a database in a dozen places. Aether turns it around: you declare that every row of some struct is saved, and what to file it under, and the game does the rest. There is no save call. You never write "this changed"; the registry finds changes by comparing, so a row that is only read is never saved.

Three steps, shown on the Purse row of meadow (message Purse { string<16> owner; u32 gold; }), which only its owner sees:

examples/meadow/Bank.h
class Bank : public server::Game {
public:
    explicit Bank(ecs::Store& store) : Game({.store = &store}) {}   // 1. give the Game a store; it must outlive the Game

    void onStart() override {
        persist<Purse>().key(&Purse::owner);                    // 2. declare: every Purse row is saved, filed under its owner
        view<Purse>().ownerOnly();
        view<Walker>().global();
    }

    void onJoin(Player& p) override {
        const std::string name = p.query("name", "anon");
        p.entity = spawn(Walker{0, 0, 60},                          // 3. what was saved under this key, or a new row
                         load<Purse>(name).value_or(Purse{String<16>(name), 0}));
    }
};

Notice that persist<Purse>() is not a call that saves a value. It says "every entity that has a Purse row is saved, the whole row, filed under its owner". After spawn you read and write the row like any other (get<Purse>(e).gold += 5;) and the game notices.

The same game runs on any of the stores below; server.cpp changes by one line.

Where rows live: stores

A store is where saved rows are kept. Three ship with the SDK, and you may write your own:

StoreRows liveUse it for
ecs::MemoryStoreIn this process only.Tests; a quick local run; "a second Game in the same process".
ecs::FileStore(path, sync = false)In memory and in a file on disk: they survive a restart of the process.One server on one machine.
ecs::RuneStore(...)In a rune broker, outside the game process, surviving its restarts and its machine. Library aether_rune, #include <aether/ecs/RuneStore.h>.A database an owner already runs. See The rune broker.
Your own ecs::StoreWherever you decide.See Your own store.
examples/meadow/server.cpp
#include "meadow/Bank.h"

int main() {
    ecs::FileStore store("meadow.db");           // or: ecs::MemoryStore store;
    Bank world(store);
    server::App app(world, {.tickRate = 20});
    return app.listen(9000) ? 0 : 1;
}

A store must outlive the Game, and is used by one Game at a time. A second Game on the same store is "the server after a restart", which is how tests restart a server.

FileStore in detail

  • The file is a log: a record per change. It is rewritten smaller when it opens if most of it is old values. A cut tail (a crash in the middle of a write) is ignored when the file is read back.
  • With sync = true each batch is also put on the disk with fsync: slower, and what a power cut needs. A crash of the process only needs the default false.
  • It is one process's file: two processes on one file is not supported.
  • The constructor throws std::runtime_error if the file cannot be created or opened, or is not a store file ("ecs::FileStore: 'game.db' is not a store file (a store file starts with AETHSTO1)"). A write that the disk refuses throws from write(); the Game reports it on stderr and tries again at the next tick.
  • It is a MemoryStore underneath, so every row is also held in memory, and its counters (Testing saves) work.

When a row is saved

Saving is lazy on purpose, so a row that changes every tick (like the meadow coin) does not become a write every tick. A row is saved:

  • some time after it changed, at most once per conflation window (default 1000 ms); a later change inside the window replaces the earlier one, and the latest value is what goes;
  • when its player leaves, at once: the rows of the player's own entity (p.entity) are saved before the entity is destroyed, so the same name joining again gets them;
  • when the server stops: everything not saved yet, whatever the window.

A new row (just spawned) is saved once, at its first window. A row that has not changed since its last save is not written again. A row that was loaded and not changed is not written back at join, at leave or at shutdown: what the store gave you is what it already has. Stepping a game by hand against a MemoryStore shows it: a row spawned at 0 ms was not saved at 250 ms, was saved at 1250 ms (saves() == 1), was not saved again at 3250 ms; a change at 3250 ms was saved by 4750 ms; and a change followed at once by a shutdown was saved by the shutdown.

Trap. Only the rows of p.entity are flushed when a player leaves. A saved row on another entity (a farm the player owns, a chest) is saved by the window and at shutdown; if you destroy that entity, its last value is pushed at the next tick whatever the window.

Change the window for one row with .conflate(window). 0ms means "at the next tick after a change":

class Chests : public server::Game {
public:
    explicit Chests(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Purse>().key(&Purse::owner).conflate(0ms);     // the default is 1000 ms
        view<Purse>().ownerOnly();
    }
};

0ms and 1s are std::chrono literals: write using namespace std::chrono_literals; once, or spell them std::chrono::milliseconds(0).

When the store is slow or fails

Saving never throws into your game. A store that fails is reported on stderr and the same rows are tried again by themselves: "aether: the store could not save (it is tried again next tick): ..." or, for a store that takes batches and then fails them, "aether: the store did not save N row(s) of 'Purse' (no answer in time | refused or not delivered); they are tried again". The same line again within 10 seconds is only counted ("(N more like it)"). At shutdown the Game waits until the store has confirmed everything, at most the store's flushTimeout() (5 s by default; a RuneStore uses its shutdownTimeout), and says what it could not confirm: "aether: the store did not confirm N row(s) within X ms at shutdown, they may be lost".

The key

Every saved row is filed under a key: one member of the struct that you name in .key(&Row::member). The key is how load finds the row again, so it is what makes a row "this player's".

RuleWhat happens if it is broken
The key member is an integer (not bool) or a String<N> (a std::string is allowed for a row with its own storage).A compile error: "persist<Row>().key(&Row::member): the key is an integer or a String<N> (or std::string) member".
A string key is at most 16 bytes (the String<16> limit of a persisted row), an integer key at most 8.On load or spawn: std::invalid_argument "'AVeryLongNameTha...' is too long: 36 bytes, at most 16 (Purse.owner; cut or check it before load/spawn)". In onJoin that refuses the client with this text as the reason.
Never zero, never empty.std::invalid_argument: "the key is empty; a persisted row's key is never empty", "the key is zero; a persisted row's key is never zero". A spawn of such a row throws too, and leaves no entity behind.
Unique among live rows.A second spawn with a key that is already online throws std::logic_error: "'Ann' is already online (the key of a Purse row is unique among live rows)". No entity is left behind.
The right kind for load.std::invalid_argument: "the key of Purse is a text (a String member), not a number" (and the reverse). An integer that does not fit a narrower member: "the key N does not fit the key member of Row".
It never changes while the row lives.At the registry level a changed key is restored from the last saved copy and never pushed (see Underneath).

load takes the key as a std::string, a String<N>, a string literal or an integer. A signed key may be negative; an integer key is stored as its little-endian bytes, as wide as the member. A string key is its text with no padding.

Tip. Cut untrusted text before using it as a key: String<16>(name.substr(0, 16)). A name that is too long throws, never truncates, and in onJoin that means that client is refused. Two players with the same name are two spawns with one key: check first (if (online.contains(name)) name = "guest" + std::to_string(++guests);) or let the refusal happen.

An integer key works the same way:

struct Score { std::uint32_t player; std::uint32_t best; };        // a saved row keyed by a number

class Ladder : public server::Game {
public:
    explicit Ladder(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override { persist<Score>().key(&Score::player); }
    void onJoin(Player& p) override {
        const std::uint32_t id = static_cast<std::uint32_t>(std::stoul(p.query("id", "1")));
        spawn(load<Score>(id).value_or(Score{id, 0}));
    }
};

Topics and views: the name of a saved row

Rows are stored under a topic: by default the struct's unqualified name ("Purse"), the same rule as a view without a name. Another name goes first: persist<Purse>("wallet"). A game with one struct per track can make a topic per track, as the kart example does (persist<BestLap>("bestlap_" + name_)).

Every topic has a second name, its by-key view: <topic>_by_<key member> unless you give .view("name"). The MemoryStore and FileStore have no views and ignore it; a RuneStore requires the view to exist and reads every saved row through it (The rune broker).

class Named : public server::Game {
public:
    explicit Named(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Purse>("wallet").key(&Purse::owner);                           // topic "wallet", view "wallet_by_owner"
    }
};
class Named2 : public server::Game {
public:
    explicit Named2(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Purse>("wallet").view("wallet_latest").key(&Purse::owner);     // topic "wallet", view "wallet_latest"
    }
};

.view("") throws std::logic_error: "persist<Purse>().view(\"\"): the view name is empty. Give it a name, persist<Purse>().view(\"Purse_latest\"), or leave view() out for the default \"<topic>_by_<key member>\"".

Loading a row: three levels

All three are optional, from the simplest up. They all answer std::optional<Row>: the row, or nothing when nothing was saved under that key.

Level 1: in onJoin

load<Row>(key) in onJoin is the code you saw on the first page. With MemoryStore and FileStore it answers at once. It throws std::logic_error if Row was not declared with persist ("the row is not persisted. Say so in onStart(): persist<Row>().key(...)") and the std::invalid_argument key errors above. It reads what the store has; a row that changed less than a moment ago was already saved when its player left, so a player who leaves and joins again gets it, even if the store has not acknowledged the save yet.

Trap. With a RuneStore this form waits for the broker: one read, at most blockingReadTimeout (3 s), and the tick stands still meanwhile. If the read cannot be answered it throws and the join is refused. Use level 2 for a player's own rows.

Level 2: before onJoin (ofPlayer and playerKey)

When a store takes time, let the Game read the player's rows before onJoin and make the client wait. Declare the rows .ofPlayer() and tell the game the key of a connecting player by overriding playerKey:

struct Hero { String<16> name; double x, y; };

class Dungeon : public server::Game {
public:
    explicit Dungeon(ecs::Store& store) : Game({.store = &store, .loadTimeout = 5s}) {}
    void onStart() override {
        persist<Hero>().key(&Hero::name).ofPlayer();
        persist<Purse>().key(&Purse::owner).ofPlayer();
        view<Hero>().global();
        view<Purse>().ownerOnly();
    }
    std::string playerKey(Player& p) override { return p.query("name"); }   // "" (the default): nothing is fetched early
    void onJoin(Player& p) override {                                         // the same code as level 1, now answered at once
        const std::string name = p.query("name");
        const String<16> key(name);
        p.entity = spawn(load<Hero>(name).value_or(Hero{key, 0, 0}), load<Purse>(name).value_or(Purse{key, 0}));
    }
};

A client with a non-empty playerKey and at least one ofPlayer row is held: it is not in players(), gets no onJoin, no input is applied, no views are sent, and what it sends waits in its input buffer. When every ofPlayer row has been read, onJoin runs in a tick and its load<Row>(thatKey) answers at once with what was fetched (the store is not asked again).

  • The key is text. playerKey returns a std::string. For a row with an integer key it must be the number as text ("42"); otherwise the client is refused: "playerKey 'abc' is not a number, but the key of Row is (playerKey returns the number as text: \"42\")".
  • The client leaves meanwhile: it is forgotten: no onJoin, no onLeave.
  • A failure refuses the client with code 4000 and the reason: a read that fails ("could not load the saved Hero of 'Ann': ..."), one that takes longer than loadTimeout (default 10 s: "loading the saved rows of 'Ann' took more than 1000 ms: the store did not answer (Game({.loadTimeout = ..}) sets the wait)"), a playerKey that throws, or a key too long for the row's key member. The server goes on.
  • An empty key is "no early fetch": onJoin runs at once, as in level 1.
  • Two clients with the same key at once are both fetched; the second spawn is refused by the duplicate-key rule. A client held while another with its key leaves asks again, so it gets the row that was just saved.
  • p in playerKey is only good for p.query(..) and p.client. Do not keep it.

Level 3: by callback, for rows that belong to no join

A guild, a shop, a chest: load<Row>(key, onLoaded[, onFailed]) sends the read and calls you in a later tick on the game thread, never inside the call:

struct Guild { String<16> name; std::uint32_t members; };

class Guilds : public server::Game {
public:
    explicit Guilds(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Guild>().key(&Guild::name);
        post([this] {                                                          // the declaration is in effect when onStart returns
            load<Guild>("Lions", [this](std::optional<Guild> g) {
                spawn(g.value_or(Guild{String<16>("Lions"), 0}));           // what was saved, or a new one
            }, [](const std::string& why) {
                std::fprintf(stderr, "guild read failed: %s\n", why.c_str()); // a failed read is never an absent row
            });
        });
    }
};

A failed read is never reported as "absent". With an onFailed you get the reason; without one a line goes to stderr and onLoaded is not called: "aether: load<Guild>('Lions') failed: the database is down (its callback was not called; give load a third argument, [](const std::string& why) { ... }, to handle it)". The same key errors as load(key) throw at the call. A callback that throws is reported ("a function given to load<Row>(key, ..) threw and was dropped: ...") and dropped. Callbacks still waiting when the server stops are dropped, not called.

Underneath, a store has readLater(topic, key, done) next to read: the default answers at once from read(), and the Game delivers every answer at the start of the next tick, whoever answered and when.

The rest of the declaration

persist<Row>() returns a builder; each method returns the builder, so you chain them after .key(..). They throw at the call (std::logic_error) for combinations that cannot work, so a mistake shows up as soon as the server starts.

CallMeaning, default
persist<Row>(topic = "")Declare. topic empty: the struct's unqualified name. Called twice for one row: std::logic_error "called twice for the same row". Takes effect when onStart returns, or at the row's first spawn, get or set, whichever is first. load does not trigger it.
.key(&Row::member)Required. See The key. A declaration with no key is an error when onStart returns: "persist<Purse>() needs a key: persist<Purse>().key(&Purse::member)".
.conflate(window)std::chrono::milliseconds. Saved at most once per window; default 1000 ms; 0ms: at the next tick after a change.
.durability(d)ecs::Durability::None | Leader | All, default Leader: what the store's acknowledgement stands for. See below. Not with .with.
.onRemove(r)ecs::OnRemove::Keep | Tombstone, default Keep: what happens to the stored record when the entity is destroyed or the row removed. See below.
.convert<Saved>(toSaved, fromSaved)Store the row as another flat struct Saved. After .key(..). See Rows that are not flat. Not with .with.
.with(storage)Your own server::Storage<Row> saves and loads the row. See Your own storage. Not with convert, durability or view.
.view(name)The name of the topic's by-key view, default <topic>_by_<key member>. Not with .with.
.ofPlayer()Fetch this row before onJoin, under playerKey(p). Needs playerKey to be overridden (otherwise nothing is fetched early). See Level 2.

Durability

The level maps to the rune producer's acknowledgement level and matters for a RuneStore: All means every record has been committed by the in-sync replicas; Leader that the partition leader has appended it; None that the producer handed it to the transport and nothing is known of the broker (a frame lost after that is lost). MemoryStore and FileStore acknowledge each batch before write returns, whatever the level.

onRemove: keep or tombstone

With Keep (default) a destroyed entity's last value is pushed and stays in the store: leaving memory does not delete the record, which is what a player's row needs (logout unloads it). With Tombstone the store's record is deleted when the entity is destroyed.

class Fleeting : public server::Game {
public:
    explicit Fleeting(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Purse>().key(&Purse::owner).conflate(0ms).durability(ecs::Durability::All).onRemove(ecs::OnRemove::Tombstone);
    }
};
Trap. Do not use Tombstone on a player's own row: the player's entity is destroyed when it leaves, and the record would be deleted at every logout. Run it and see: with Tombstone a destroyed entity left erases() == 1 and an empty topic. Use it for rows that should disappear with their entity (a dropped item, a closed match).

Rows that are not flat

A saved row goes into a store as a fixed-layout record, so the struct must be flat: numbers, bool, String<N>, nested structs and std::arrays of those. Two things people expect to work do not, and the reason matters:

  • ecs::List<T, N>: its capacity would be baked into the database.
  • ecs::Entity: an entity id means nothing after a restart. Store the other entity's own key (an integer or a string) instead.

Declaring such a row without saying how to store it fails when the server starts, with the way out in the message: "persist<Bag>(): Bag has a List or an Entity member (or is not flat), so it cannot be saved as it is. Say how to store it: persist<Bag>().key(&Bag::member).convert<Saved>(toSaved, fromSaved), where Saved is a flat struct you write, or persist<Bag>().key(&Bag::member).with(yourStorage), a server::Storage<Bag> that saves the whole row". A struct with a std::map, std::vector or std::string gets the similar message that offers only .with(yourStorage) (and convert, see below).

convert<Saved>: save it as another struct

Write a flat Saved struct and two functions. The live row stays whatever it is; what goes into the store, and comes back at load, is Saved.

struct Item { std::uint16_t id; std::uint16_t count; };
struct Bag      { String<16> owner; ecs::List<Item, 4> items; };        // live: a List
struct SavedBag { String<16> owner; std::array<Item, 4> items; std::uint8_t used; };   // saved: flat

SavedBag toSaved(const Bag& b) {
    SavedBag s{b.owner, {}, 0};                                          // toSaved copies the key into Saved
    for (const Item& i : b.items) s.items[s.used++] = i;
    return s;
}
Bag fromSaved(const SavedBag& s) {
    Bag b{s.owner, {}};
    for (int i = 0; i < s.used; ++i) b.items.push(s.items[i]);
    return b;
}

class Bags : public server::Game {
public:
    explicit Bags(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Bag>().key(&Bag::owner).convert<SavedBag>(toSaved, fromSaved);
        view<Bag>().ownerOnly();
    }
};
Rule of convert<Saved>(toSaved, fromSaved)Detail
Call it after .key(..)Otherwise std::logic_error: "give the key first: persist<Bag>().key(&Bag::member).convert<Saved>(..)".
Saved is flatA compile error says so and suggests turning the std::map/std::vector into a fixed number of members (a std::array<T, N> and a count, a String<N>), or keeping the whole row in your own storage.
Saved has a member named like the keytoSaved must copy the key into it. fromSaved need not: the row's own key is always the one stored.
toSaved is Saved f(const Row&)It runs when a changed row is saved. Lambdas may capture (this, for a clock or a table of names); plain functions work; an overloaded function name does not.
fromSaved is Row f(const Saved&)It runs when a stored row is read (load, ofPlayer). It must return the row itself; the compiler tells you if it returns Saved or something else.
The topic's layout is Saved'sThe store only ever sees Saved; the layout check (below) is about Saved.
A row that cannot be flat at all (a std::map, a std::vector)Works with the same call. The live row becomes a server-only component; it must be copyable and have bool operator==(const Row&) const = default; so the game can tell whether it changed. The key must be a direct member and the row a plain struct (no constructors, no base class). load<Row>(key) in one call then works only with a store that answers at once (MemoryStore, FileStore); with a RuneStore use .ofPlayer() or the callback form.
Not with .with(..)std::logic_error: "convert<Saved>(..) and with(storage) cannot be used together ...".

Server-only rows (local components)

A row that is not a flat struct (anything with a std::string, a std::vector, a std::map) is a local component: it lives on the server, is never sent to a client and is not saved by default. It can be saved in the same two ways: convert<Saved> above, or .with(storage) below, where your storage keeps the whole row. A row with its own storage is a local component and so not a view row; to show part of it to clients, copy what is shown into a flat row (a function view, see Data). Views themselves are never saved: only components are.

Time restarts: save what is left, not the clock

time() starts again at zero when the server restarts, so an absolute game time saved in a row is wrong after a reload. Save the time that is left and rebuild it at load (the farm example does exactly this). If the thing should keep growing while the server is down, save the wall clock instead (std::chrono::system_clock, as seconds in a double).

struct Plot      { String<16> owner; double ripeAt; };           // live: an absolute game time
struct SavedPlot { String<16> owner; double remaining; };        // saved: seconds left

class Plots : public server::Game {
public:
    explicit Plots(ecs::Store& store) : Game({.store = &store}) {}
    void onStart() override {
        persist<Plot>().key(&Plot::owner).convert<SavedPlot>(
            [this](const Plot& p) { return SavedPlot{p.owner, std::max(0.0, p.ripeAt - time())}; },
            [this](const SavedPlot& s) { return Plot{s.owner, time() + s.remaining}; });
    }
};

Your own storage: with(storage)

When a row belongs in a database of your own (SQL, a key-value service), keep the game's when and supply the how. The Game still sees the row change, waits out the conflation window, saves when the player leaves and when the server stops, fetches the row before onJoin (ofPlayer) and answers load. Your server::Storage<Row> decides where the bytes go.

struct Stash {
    String<16> owner;
    std::map<std::string, int> things;                          // not flat: fine for a row with its own storage
    bool operator==(const Stash&) const = default;             // required: the Game compares the row with the copy it last saved
};

class StashStorage : public server::Storage<Stash> {
public:
    void save(const Stash& row) override { db_[row.owner.str()] = row; }                // changed (after the window), left, or shutdown
    void erase(const Stash& row) override { db_.erase(row.owner.str()); }               // only with onRemove(Tombstone)
    void load(server::Key key, server::Loaded<Stash> done) override {
        const auto it = db_.find(std::string(key.text()));
        if (it != db_.end()) done(it->second); else done(std::nullopt);                // or done.fail("the database is down")
    }
    void poll() override {}                                                                // once a tick: deliver finished work here
private:
    std::map<std::string, Stash> db_;
};

class Stashes : public server::Game {
public:
    explicit Stashes(StashStorage& storage) : storage_(storage) {}
    void onStart() override { persist<Stash>().key(&Stash::owner).conflate(1s).with(storage_); }   // no Game({.store}) needed
    void onJoin(Player& p) override {
        const std::string name = p.query("name", "anon");
        p.entity = spawn(load<Stash>(name).value_or(Stash{String<16>(name), {}}));
    }
private:
    StashStorage& storage_;                                      // must outlive the Game
};
ItemMeaning
save(const Row&)The row changed (and its window passed), its player left, or the server stops. The whole row. May throw: reported on stderr (once per 10 s per problem) and offered again at the next window; a row saved when its player leaves or at shutdown is offered once.
erase(const Row&)The row's entity was removed and the row has onRemove(Tombstone): the record should go. Default: nothing.
load(Key, Loaded<Row>)Answer with exactly one of done(row), done(std::nullopt) (nothing under that key) or done.fail("why"), now or in a later tick (keep done, answer from poll()). A throw is done.fail(what()). A second answer is ignored. A row that left a moment ago has already been given to save().
poll()Once per tick on the game thread: deliver finished work (the done of a slow load, the result of a save you queued). Default: nothing.
server::Keytext() (a text key, or the decimal text of a number), isNumber(), number() (throws std::invalid_argument for a text key; a negative key of a signed member reads back by a cast), and it converts to std::string_view, so db.get(key) works for a text key.

How the Game knows the row changed

A .with row has to be compared, so it needs operator== (= default is enough when its members can be compared); without it the build stops with "a row with its own storage ... needs operator==, so that the Game can tell whether it changed and a read never causes a save. Add to the row: bool operator==(const Row&) const = default;". set(e, row) and a writable get<Row>(e) mark the row; at the end of the window the Game compares it with the copy it last handed to your storage and calls save only for a real difference. get<Row>(e, READONLY) does not even mark it, for example: a spawned row was saved once, a read changed nothing, a change saved it again.

  • A spawned row is handed to the storage once, at its first window, unless it came from a load of the same key (load, the callback form or ofPlayer): what was loaded is what the storage already has, and the row is saved only after a real change.
  • Changing a row through a reference you kept across ticks, or through registry(), is not seen until the player leaves; a leave and the shutdown always save.
  • load<Row>(key) in one call works only if the storage answers inside its load; otherwise it throws and says to use the callback form or ofPlayer().
  • A game whose rows are all .with needs no Game({.store}).
  • These combinations throw at the call: .with after convert/durability/view, and each of those after .with. One message per case: "convert<Saved>(..) and with(storage) cannot be used together ...", "durability(..) has no meaning with with(storage): the Game does not save the row, your storage does ...", "view(..) has no meaning with with(storage): a row with its own storage has no topic and no view".

The storage and everything it calls run on the game thread. A storage that works on a thread of its own hands finished work over in poll().

Changing a saved struct

Each topic remembers the layout of the row it was saved with. If your struct no longer has the same members (one added, removed, reordered or widened), persist throws when the game starts and the server does not start. Nothing is migrated for you: old rows are never read into the wrong fields.

The error lists both layouts. Here Pot gained gems:

ecs::Store: the topic 'pot' was saved with another layout of its row than the one this game has now (a member was added,
removed, reordered or resized), so its saved rows cannot be read into the new struct. Saved: [owner:16B, gold:4B].
Now: [owner:16B, gold:4B, gems:4B]. (fingerprints 9776dc061e80d171 and fa2d7217d9568d6f). Put the old struct back, or
save the new one under a new topic name (persist<Row>("newname", ...)), or delete the store to start empty

Your three ways out: put the old struct back; save the new struct under a new topic name (persist<Pot>("pot2").key(&Pot::owner)) and copy the old rows yourself; or delete the store. Members of 8 or 16 bits and float32 are saved as they are. A store of your own that does not override expect loses this protection: see Your own store.

Declaring early: persist before the first spawn

Call persist<Row>() in onStart, before anything spawns, gets, sets or views that row as a plain row. If the row was already registered as a plain row the declaration throws: "server::Game::persist<Row>(): the row is already registered as a plain row (by spawn, get or set); call persist<Row>() before the first spawn of the row (first in onStart())". A Game with a persisted row and no store (and no .with) throws at start: "this Game has no store. Give it one: Game({.store = &store}) with an ecs::MemoryStore or ecs::FileStore that outlives the Game (or give the row your own storage: persist<Row>().key(..).with(yourStorage))". An exception out of onStart means the world never ticks and App::listen returns false.

Trap. load in onStart, right after the persist call, throws "server::Game::load<Row>: the row is not persisted. Say so in onStart(): ...", because the declaration only takes effect when onStart returns (a spawn of the row does make it earlier). A world row (a chest, a door) is saved the same way as a player's, but read it from a post(..) function (it runs at the start of the first tick) or in a callback load as in the guild example above, not directly in onStart.

Testing saves

A MemoryStore counts what the game writes to it, read-only: saves() (rows put, new or changed) and erases() (rows removed). Reading does not count, and neither does a FileStore reading its file when it opens. That is enough to prove "nothing was saved" without a subclass:

examples/meadow/BankTest.cpp
TEST(Bank, JoiningAndLeavingWithoutAChangeSavesNothing) {
    ecs::MemoryStore store;
    test::Server<Bank> server(test::Config{.rate = {.perSecond = 0}}, store);
    const auto before = store.saves();
    {
        client::Session a(server.url("name=Ann"));
        a.connect();
        ASSERT_TRUE(test::pump(a, [&] { return a.ready(); }));      // Ann joins, changes nothing, leaves
    }
    ASSERT_TRUE(test::waitFor([&] { return server.run([](Bank& w) { return w.players().empty(); }); }));   // the leave was handled
    EXPECT_EQ(store.saves(), before);                                    // so nothing was saved
}

A new player's rows are saved once, at their first window: wait for store.saves() == 1 before you take before if the test creates one. To test a restart, destroy the test::Server (that saves everything), start a new one on the same store (for a file, a new FileStore on the same path), and the player has its row back. With the meadow World a player earns coins for 300 ms and leaves; after the restart the same name has them back.

examples/meadow/RestartTest.cpp
TEST(Meadow, GoldSurvivesARestart) {
    ecs::MemoryStore store;
    {
        test::Server<World> server(store);
        {
            client::Session a(server.url("name=Ann"), {.contract = meadow::contract});
            a.connect();
            ASSERT_TRUE(test::pump(a, [&] { return a.ready(); }));
            test::pumpFor(300ms, a);                                     // a coin a tick
        }
        ASSERT_TRUE(test::waitFor([&] { return server.run([](World& w) { return w.players().empty(); }); }));
    }                                                                    // the server stops here: everything left is saved
    EXPECT_EQ(store.size("Purse"), 1u);
    test::Server<World> again(store);
    client::Session a(again.url("name=Ann"), {.contract = meadow::contract});
    a.connect();
    ASSERT_TRUE(test::pump(a, [&] { return a.ready(); }));
    const auto gold = again.run([](World& w) { return w.get<meadow::Purse>(w.players().begin()->entity, READONLY).gold; });
    EXPECT_GT(gold, 0u);
}

The store may be read from the test thread while the server runs: store.size("Purse"), store.read("Purse", keyBytes), store.each("Purse", fn) (a string key is its text, an integer key its little-endian bytes). Testing documents test::Server, pump and waitFor.

Your own store

A store takes the changed rows in batches and hands a record back by key. Derive from ecs::Store when the rows should go somewhere else than a file or a broker. Three functions are required (write, read, each); the rest have defaults.

examples/meadow/MapStore.h
class MapStore : public ecs::Store {
public:
    bool write(const ecs::Batch& b, ecs::Persistence& p) override {      // true: the batch is taken
        ecs::eachRecord(b, [&](std::span<const std::uint8_t> key, std::span<const std::uint8_t> value) {
            auto& topic = topics_[std::string(b.topic)];
            const std::string k(key.begin(), key.end());
            if (value.empty()) topic.erase(k);                              // an empty value is a tombstone
            else topic[k].assign(value.begin(), value.end());
        });
        p.acknowledge(b.sequence);                                          // done before returning: acknowledge now
        return true;
    }
    void expect(std::string_view topic, std::uint64_t fingerprint, std::string_view layout) override {
        auto [it, fresh] = layouts_.try_emplace(std::string(topic), fingerprint);
        if (!fresh && it->second != fingerprint)                          // refuse a changed struct
            throw std::runtime_error("topic '" + std::string(topic) + "' has another layout: " + std::string(layout));
    }
    std::optional<std::vector<std::uint8_t>> read(std::string_view topic, std::span<const std::uint8_t> key) const override {
        const auto t = topics_.find(std::string(topic));
        if (t == topics_.end()) return std::nullopt;
        const auto r = t->second.find(std::string(key.begin(), key.end()));
        if (r == t->second.end()) return std::nullopt;
        return r->second;
    }
    std::size_t each(std::string_view topic, const RecordFn& fn) const override {
        const auto t = topics_.find(std::string(topic));
        if (t == topics_.end()) return 0;
        for (const auto& [k, v] : t->second) fn({reinterpret_cast<const std::uint8_t*>(k.data()), k.size()}, v);
        return t->second.size();
    }
private:
    std::map<std::string, std::map<std::string, std::vector<std::uint8_t>>> topics_;
    std::map<std::string, std::uint64_t> layouts_;
};
Trap. A store of your own that does not override expect loads a record of a struct that changed to a same-size layout into the wrong members, because the record itself carries no layout (only a different length is noticed). Override it.

Every function of the ecs::Store interface is listed in the reference below.

The rune broker

When rows must live in a database that the owner already runs, ecs::RuneStore keeps them in a rune broker. The game is written exactly as with a FileStore; only server.cpp changes (link aether_rune; a game that does not persist through rune links no rune or Aeron code):

ecs::RuneStore store("localhost:40200");          // the broker's address; nothing else is needed (a broker in dev mode)
Bank world(store);                                // Bank(ecs::Store& store) : Game({.store = &store}) {}
server::App app(world, {.tickRate = 20});

The constructor does nothing on the network. The rune client is started and connected when the Game declares its first row (persist<Row>() in onStart, on the world thread), and it is closed at the Game's shutdown or when the store is destroyed. Nothing is loaded at start and no saved row is held in memory: a row is read when it is asked for, through the topic's by-key view.

Three ways to make one

ecs::RuneStore a("broker.internal:40200");                              // 1. address, optionally adjusting any option
ecs::RuneStore b("broker.internal:40200", [](ecs::RuneStoreOptions& o) {
    o.create = true;                                                    //    local development: define a missing topic and view
    o.partitions = 4;
    o.readTimeout = 5s;
    o.client.egressChannel = "aeron:udp?endpoint=game-1.internal:40400";  //    when the broker is on another machine
});
ecs::RuneStore c({.broker = "broker.internal:40200", .user = "world1",   // 2. address and login
                  .passwordFile = "/run/secrets/world1.pw", .tlsCaFile = "/run/secrets/ca.pem"});
ecs::RuneStore d(ecs::RuneStoreOptions{});                              // 3. all the options by hand (gatewayChannel, egressChannel, dataDir must be set)

The defaults of the address form: the topic and its view must exist (create = false); the client endpoint for the broker's answers is UDP port 40400 on the local machine; a data directory for rune's embedded driver under the temporary directory (aether-rune-<pid>-<n>); a connection timeout of 15 s.

Options

OptionType, defaultMeaning
broker (RuneConnection)std::string"host:port" of the broker's client endpoint, or a full aeron: channel. Empty throws std::invalid_argument ("the broker address is empty").
userstd::string, emptyThe rune user this server logs in as; empty: no login (a broker in dev mode, RUNE_AUTH=disabled).
password, passwordFilestd::string, emptyThe password: preferably in a file (passwordFile, mode 0600 or 0400, checked when the store is made), or the text itself (password: kept off logs and messages, but not off the command line or the binary). Give one, not both.
tlsCaFilestd::string, emptyPEM file of the CA that signed the broker's certificate; empty: the system's trusted CAs.
adjuststd::function<void(RuneStoreOptions&)>Any other option, as in the address form.
createbool, falseDefine a missing topic (and a Game row's by-key view) from the row's type: for local development and tests. A mistyped topic name makes a new empty topic; the schema check still protects every topic that exists. A view that exists is never replaced.
partitions, replicationFactorstd::int32_t 1, std::int16_t 1For topics the store creates.
openTimeoutmilliseconds, 15000Connection timeout.
connectRetrymilliseconds, 0The broker may be down when the server starts: keep trying to connect and log in for up to this long (a short backoff between attempts, one stderr line per attempt), then fail with the same error as without it. 0 fails at once. Only an unreachable broker is retried; a login the broker refuses fails at once.
shutdownTimeoutmilliseconds, 10000How long the Game's shutdown waits for the broker's acknowledgements (this is the store's flushTimeout()).
readTimeoutmilliseconds, 10000A read by key that the view cannot answer yet is retried for this long, then fails with the reason. The Game's loadTimeout bounds it too.
blockingReadTimeoutmilliseconds, 3000The same for the blocking read() (level 1), but the tick waits, so it gives up sooner.
completionsBytesstd::size_t, 1 MiBCapacity of the completion queue: a power of two, 4096 to 1 GiB; each record in flight takes 48 bytes of it. Must hold at least persistence.maxBatchRecords records (the constructor checks).
backPressurePumpsint, 64How many times to pump the client when the producer is back-pressured inside a batch.
client, producer, persistencerune::client::ClientConfig, rune::client::ProducerConfig, ecs::PersistenceOptionsThe rune client (gatewayChannel, egressChannel, dataDir, ...; its run mode is forced to Invoker and reconnectForeverAfterLive to true), a template for the producers (acks is set per durability level), and the batch limits and in-flight timeout of the persistence underneath.

A user with no password, a password with no user, both password forms, a missing password file, a file that group or others can read, or an unreadable CA file all throw std::invalid_argument when the store is made, saying what to set ("the password file '...' can be read by others: chmod 600 it ..."). The login is SCRAM-SHA-512 over TLS: the client library never sends the password, and a client with credentials refuses a plaintext session. The password never appears in a message.

What an admin creates, once per saved row

The store does not make topics unless create = true. A missing topic or view is a start-up error that names the row, topic and view and prints the statement to run. For persist<Purse>("wallet") they are:

rune ql --broker localhost:40200 -e "CREATE TOPIC wallet { owner string(16), gold uint32 } KEY owner WITH (cleanup.policy = compact);"
rune ql --broker localhost:40200 -e "CREATE VIEW wallet_by_owner KEY owner AS SELECT gold FROM wallet;"
  • The topic must be compacted (cleanup.policy = compact, set at creation: rune cannot alter it later), with no time or size retention. It may be named as you like: rune's fingerprint includes the schema name, which the CLI sets to the topic's name, and the store compares the layout and ignores the name.
  • The view is the identity view of the topic: the same key, every other member, nothing computed. It is named <topic>_by_<key member> unless the row says .view("name"). A view that exists but is another one (another topic's, a projection that lacks a member, an aggregate) is a start-up error that says what differs and how to replace it (DROP VIEW name; and the statement above): the store never replaces a view.
  • A row with an enum member cannot be written in rune QL; the message then says to set create = true once (development) or to make the topic from the rune client.
  • A topic that is autoincrement (partitioner Modulo) is refused: rows are written by key, as upserts.

Logging in and grants

A broker as it is meant to run requires a login (SCRAM over TLS, every request checked against grants). Without .user the store connects without a login, right for a broker in dev mode; a broker that requires one refuses it at start with the call site to use. Credentials given to a dev broker (which serves no TLS) are refused, not ignored: connect to it without .user. Every refusal stops the server at start (persist<Row>() throws a RuneStoreError) and names the user, never the password: wrong password, unknown user and disabled user (the broker answers all three the same on purpose); a one-time password (the message gives the ALTER USER statement); a missing grant (which operation on which topic, and the GRANT to run).

Least privilege for a world server whose topics and views are all named world_..., run once as a security admin:

CREATE USER world1 WITH PASSWORD '<12 or more characters>';
GRANT WRITE    ON TOPIC PREFIX 'world_' TO USER world1;   -- saving rows (and DESCRIBE of the topics, which any grant implies)
GRANT READ     ON VIEW  PREFIX 'world_' TO USER world1;   -- loading a row: the read of a key goes through the view
GRANT DESCRIBE ON CLUSTER TO USER world1;                 -- opening a view reads rune's catalog (all or nothing: it shows every object name)
-- only for create = true (development):
GRANT CREATE   ON TOPIC PREFIX 'world_' TO USER world1;
GRANT CREATE   ON VIEW  PREFIX 'world_' TO USER world1;   -- and READ ON TOPIC PREFIX 'world_': a view's creator reads its source

READ on the topics is not needed: nothing subscribes. While the game runs, REVOKE WRITE makes the next save fail with the same message, once per 10 s on stderr, and the rows wait; GRANT it again and they are sent. A session the broker ended (the user dropped or disabled, or its password reset with TERMINATE SESSIONS) is not reconnected by rune's client: the store reports it and logs in again with a new client every 10 s until it works.

When the broker is not there

SituationWhat happens
Down when the game startspersist<Row>() throws a RuneStoreError ("RuneStore::expect: connecting to ...") after the connection timeout, so onStart fails and the server does not start. (Starting without the broker would give every player a new row and overwrite the saved ones when the broker came back.) Start the broker first, or set connectRetry.
Lost during the gameThe client reconnects by itself, for ever. The game keeps running. A read by key is retried until readTimeout and then fails (a failed load, never an absent row). Changed rows stay dirty and their batches fail after the producer's delivery timeout (default 10 s in rune): one stderr line per kind of problem ("the store did not save N row(s) of 'X' ..."). Nothing throws into the game.
BackThe rows that were not delivered are sent again by themselves; nothing is lost while the game process lives. A game stopped while the broker is down loses the rows it could not deliver, and says so at shutdown.
View missing, or not the identity viewpersist<Row>() throws, naming the row, topic, view, what differs and the statement to run.
Topic schema differs from the rowpersist<Row>() throws, listing the saved and the present members, as the file does.

Blocking and non-blocking loads

  • load<Row>(key, fn) and the ofPlayer() prefetch send the read and the answer comes from poll() on the world thread, a tick later: nobody waits.
  • load<Row>(key) outside a prefetch sends the read and pumps the client until it is answered, at most blockingReadTimeout: the tick waits (about one round trip).
  • An answer of NotReady (the view still catches up after a restart, or has not folded a write yet), Busy, Throttled or NotLeader is asked again (10 ms, doubling, at most 250 ms apart) until readTimeout, then the read fails with the reason. NotAuthorized fails with the GRANT to run.
  • Read your writes: the store keeps, per partition of each topic, the highest offset the broker acknowledged for a record of this store, and passes it as a floor: the view answers NotReady until it has folded it, so a row this process saved is never read back older. A row that left memory and is not acknowledged yet is answered by the Game itself, not the broker.

Saves are asynchronous: write() hands the batch to rune's producer, and the Game polls the store every tick, which collects the acknowledgements. Members of 8 or 16 bits and float32 are stored as they are (rune-client-cpp 0.1.7, broker at cluster feature level 9, rune 0.4.0). Every function of the store runs on the world thread.

Underneath: registry-level persistence

A Game is built from public pieces, and so is its saving. persist<Row>() declares an ecs::PersistOptions on the registry's component; an ecs::Persistence watches the changed rows and pushes them in batches to a sink (a store's write); the store acknowledges a batch, and only then are its rows clean. This is the layer you use when you write a world by hand (see ECS), and it is what the rules above are made of. Everything here runs on the world thread.

examples/meadow/ByHand.cpp
struct Hero { std::uint32_t id; double x; };

void byHand() {
    ecs::MemoryStore store;
    ecs::Registry registry;
    const auto hero = registry.component<Hero>(ecs::persist("hero", &Hero::id).conflate(0ms));   // the topic, and the member that is the key
    ecs::Persistence persistence(registry, [&](const ecs::Batch& b) { return store.write(b, persistence); });

    const auto e = registry.create();
    registry.add(e, hero, Hero{7, 1.5});          // the only way to add a persisted component: with its key
    registry.commit(1, 100);                      // the tick and the time in ms
    store.poll(persistence);                      // each tick: the store reports outcomes ...
    persistence.collect();                        // ... and the changed rows are pushed
    // store.size("hero") == 1; persistence.stats().batchesAcknowledged == 1

    ecs::Registry fresh;                          // "the server after a restart"
    const auto hero2 = fresh.component<Hero>(ecs::persist("hero", &Hero::id));
    const std::uint8_t key[4] = {7, 0, 0, 0};     // an integer key is its little-endian bytes
    const auto record = store.read("hero", key);
    const auto again = fresh.create();
    fresh.load(again, hero2, *record);            // a record goes into a row; the row is clean
}

Rows are written in place, so there is no write hook: collect compares each persisted row with a shadow copy, the value last pushed. A write that leaves a row unchanged is no change, and a value that goes back to what was last pushed stops being dirty. This costs one extra copy of each persisted component and one compare per row per collect.

Row states

  • Dirty from the first collect that finds a row different from its shadow (a new row counts as different). It is due when now - thatTime >= the component's conflate; a due row is pushed with its value at that moment. Later writes inside the window replace earlier ones.
  • In flight: collect cuts due rows into batches (maxBatchRecords or maxBatchBytes, either), numbers each batch (increasing sequence) and calls the sink. Rows stay in flight until acknowledge(sequence). A row written while in flight is dirty again, and pushed only after the batch in flight has its outcome, so pushes of one key stay in order.
  • Clean once the batch is acknowledged.
  • fail(sequence), or a batch with no outcome after inFlightTimeoutMs, makes its rows dirty again and due at once; they go out with their latest value. A late acknowledge of a timed-out batch returns false: delivery is at least once.
  • A sink that returns false could not take the batch: the rows stay dirty and due, nothing else is sent in this collect (that is the only back-pressure), and the same rows are tried at the next collect. A sink that throws did not take it either; the exception goes to the caller of collect.

When a row leaves memory (destroy, remove, or erase), its value is kept for pushing in a staging area and pushed at the next collect, whatever the window. With OnRemove::Tombstone an empty value (a tombstone) is pushed instead, replacing a dirty value that was not pushed yet. erase(entity, component) writes a tombstone whatever OnRemove says, and removes the component (false if the entity has none). A persisted component with no Persistence attached makes the first registry.commit throw std::logic_error naming the components. A registry with no persisted component needs none.

Rules: one Persistence per registry, destroyed before the registry; do not change structure (destroy, add, remove) from the sink or the error handler; a row is at most 65535 bytes. A value in a batch is [blockLength u16][version u16][row], little-endian, and an empty value is a tombstone; the records of a batch are [keyLen u16][key][valueLen u32][value] ....

Reference

Game::persist and the builder

template <class S> PersistBuilder<S>& persist(std::string topic = {});
// PersistBuilder<S>
template <class T> PersistBuilder& key(T S::* member);
PersistBuilder& conflate(std::chrono::milliseconds window);
PersistBuilder& durability(ecs::Durability d);
PersistBuilder& onRemove(ecs::OnRemove r);
template <class Saved, class ToSaved, class FromSaved> PersistBuilder& convert(ToSaved toSaved, FromSaved fromSaved);
PersistBuilder& with(Storage<S>& storage);
PersistBuilder& view(std::string name);
PersistBuilder& ofPlayer();
CallParametersReturns, throws
persist<S>(topic)S: the row struct. topic: the topic name; empty means the struct's unqualified name.The builder (a reference that lives as long as the Game). std::logic_error "called twice for the same row". Declared in onStart; made when it returns or at the row's first use. Errors at that time: no key, no store (without .with), row already registered as a plain row, non-flat row without convert/with, a layout the store refuses.
key(&S::member)An integer (not bool) or String<N> (or std::string with with) member.The builder. Compile error for another type. Required.
conflate(window)window: std::chrono::milliseconds, default 1000 ms; 0 = next tick.The builder.
durability(d)d: None, Leader (default), All.The builder. std::logic_error if with was called.
onRemove(r)r: Keep (default), Tombstone.The builder. With with, Tombstone calls your erase.
convert<Saved>(toSaved, fromSaved)Saved: flat struct with a member named like the key. toSaved: Saved(const Row&). fromSaved: Row(const Saved&).The builder. std::logic_error if called before key or after with; compile errors for wrong signatures, a non-flat Saved, a row without operator==/copy (non-flat rows).
with(storage)storage: a server::Storage<S>& that outlives the Game. S must be copyable with operator==.The builder. std::logic_error after convert, durability or view.
view(name)name: non-empty.The builder. std::logic_error for an empty name or with with.
ofPlayer()-The builder. Fetch before onJoin under playerKey(p).

Game::load, playerKey, rememberedLoads

template <class S, class K> std::optional<S> load(const K& key);
template <class S, class K, class OnLoaded> void load(const K& key, OnLoaded onLoaded);
template <class S, class K, class OnLoaded, class OnFailed> void load(const K& key, OnLoaded onLoaded, OnFailed onFailed);
virtual std::string playerKey(Player&);
std::size_t rememberedLoads() const;
ItemParametersReturns, throws
load<S>(key)key: std::string, String<N>, string literal or integer, of the row's key kind.The row or std::nullopt. std::invalid_argument for an empty, zero, too long or wrong-kind key; std::logic_error if S is not persisted, or the storage of a .with row answers later; std::runtime_error "the storage could not load it: ..." if it answered fail. Inside onJoin, for an ofPlayer row and the key playerKey gave, it answers from what was fetched. With a RuneStore it blocks the tick (at most 3 s) and throws when the read cannot be answered.
load<S>(key, onLoaded)onLoaded: callable as (std::optional<S>).Nothing. Called on the game thread in a later tick, never inside the call. A failed read without onFailed prints one stderr line and does not call onLoaded. Same key errors, thrown at the call.
load<S>(key, onLoaded, onFailed)onFailed: callable as (const std::string& why).Nothing. onFailed is called with the reason when the read fails.
playerKey(p)p: only query(..) and client are good.A non-empty key with at least one ofPlayer() row holds the client until the rows are fetched; empty (default): nothing fetched early. A throw refuses the client.
rememberedLoads()-For tests: how many loaded values are remembered for a spawn: one per key that a load answered and no row was spawned for yet. An entry goes when its row is spawned or saved, and after a minute of the game's clock.

GameOptions fields for saving

FieldType, defaultMeaning
storeecs::Store*, nullptrWhere persisted rows are kept. Needed for persist unless every row has its own storage. Outlives the Game; one Game at a time.
loadTimeoutstd::chrono::milliseconds, 10 sHow long a client may wait, not yet joined, for the ofPlayer rows. Past it the client is refused (4000) and the server goes on.

server::Storage, Key, Loaded

template <class Row> class Storage {
public:
    virtual ~Storage() = default;
    virtual void save(const Row& row) = 0;
    virtual void erase(const Row& row) {}
    virtual void load(Key key, Loaded<Row> done) = 0;
    virtual void poll() {}
};
class Key {
public:
    explicit Key(std::string text, bool numeric = false);
    std::string_view text() const;   operator std::string_view() const;
    bool isNumber() const;   std::uint64_t number() const;
};
template <class Row> class Loaded {
public:
    Loaded(std::function<void(std::optional<Row>)> found, std::function<void(std::string)> failed);
    void operator()(std::optional<Row> row) const;
    void fail(std::string why) const;
};

Loaded is copyable and may be kept until you answer; call exactly one of done(row), done(std::nullopt), done.fail("why"), once; a second call is ignored. The Game makes these; a test of a storage can make one with the constructor to see what it answers. Key(text, numeric): numeric says the row's key is an integer whose decimal text is text. A storage's save, load, erase and poll are all called on the game thread.

ecs::Store

class Store {
public:
    using RecordFn = std::function<void(std::span<const std::uint8_t> key, std::span<const std::uint8_t> record)>;
    virtual bool write(const Batch& batch, Persistence& persistence) = 0;
    virtual void poll(Persistence& persistence);
    virtual void expect(std::string_view topic, std::uint64_t fingerprint, std::string_view layout);
    virtual void expect(std::string_view topic, const Type& stored, std::string_view layout);
    virtual void expectView(std::string_view topic, std::string_view view, std::string_view keyMember);
    virtual std::chrono::milliseconds flushTimeout() const;
    virtual void detach(Persistence& persistence);
    virtual std::string lastError() const;
    virtual std::optional<std::vector<std::uint8_t>> read(std::string_view topic, std::span<const std::uint8_t> key) const = 0;
    virtual void readLater(std::string_view topic, std::span<const std::uint8_t> key, std::function<void(ReadResult)> done);
    virtual std::size_t each(std::string_view topic, const RecordFn& fn) const = 0;
};
struct ReadResult { enum class Kind : std::uint8_t { Found, Absent, Failed }; Kind kind; std::vector<std::uint8_t> record; std::string error;
                    static ReadResult found(std::vector<std::uint8_t>); static ReadResult absent(); static ReadResult failure(std::string why); };
void eachRecord(const Batch& batch, const std::function<void(std::span<const std::uint8_t> key, std::span<const std::uint8_t> value)>& fn);
FunctionContract
write(batch, persistence)The sink of a Persistence. Return true: the batch is taken. A store that keeps it in a queue answers true and calls persistence.acknowledge(batch.sequence) or fail(..) from poll() when it is stored. False: it could not take it now; the rows stay dirty and are offered again. Memory and File stores finish before they return and have acknowledged. May be called from any thread (a store locks); a Persistence is the world thread's.
poll(persistence)Once per tick, before persistence.collect(): report the outcome of earlier batches. Default: nothing.
expect(topic, fingerprint, layout)Called when the game starts, once per topic. The first time a topic is named its fingerprint (and layout, a text of the members, for the message) is remembered; afterwards a different fingerprint throws std::runtime_error saying what changed and what to do. The default accepts everything: override it.
expect(topic, type, layout)The same with the saved type, for a store that keeps its rows in a database and needs it to define the topic. Default calls the one above. A store that overrides only this one (RuneStore) also overrides the one above.
expectView(topic, view, keyMember)Called right after expect with the by-key view's name and the key member. A store that reads through views (RuneStore) requires it to exist. Default: nothing (Memory and File have no views).
flushTimeout()How long the Game waits at shutdown for what it handed over to be acknowledged. Default 5 s.
detach(persistence)The Game is done with this Persistence (after the last flush): release state kept for it. The store may be given to a new Game afterwards.
lastError()The last problem in words, or empty; the Game adds it to its messages.
read(topic, key)The record [blockLength u16][version u16][row] of one key, or nothing. Throwing is a failed read. A String key is its text with no padding; an integer key its little-endian bytes, as wide as the member.
readLater(topic, key, done)The same read, answered later: done(ReadResult) exactly once, on the world thread, inside this call or from a later poll(). Default: answers at once from read(); a throw is a Failed answer. The key bytes are only valid during the call.
each(topic, fn)fn(key, record) for each record of a topic (the latest of each key). Returns how many. fn must not call the store.

ecs::MemoryStore and ecs::FileStore

class MemoryStore : public Store {
public:
    std::size_t size(std::string_view topic) const;
    std::size_t saves() const;   std::size_t erases() const;
    // write, read, each, expect: as Store
};
class FileStore : public MemoryStore {
public:
    explicit FileStore(std::filesystem::path path, bool sync = false);
    const std::filesystem::path& path() const;
};
ItemMeaning
size(topic)How many keys a topic holds.
saves(), erases()What the game has written since the store was made: rows put (new or changed) and rows removed. Reading does not count, nor does a FileStore reading its file at open.
FileStore(path, sync)path: the file, created if absent. sync: also fsync each batch. Throws std::runtime_error for a file that cannot be opened or is not a store file. Not copyable.
path()The path given.

ecs::RuneStore

explicit RuneStore(std::string_view brokerAddress, const std::function<void(RuneStoreOptions&)>& adjust = {});
explicit RuneStore(RuneConnection connection);
explicit RuneStore(RuneStoreOptions options);
RuneStore(Registry& registry, RuneStoreOptions options);      // by hand
void open();   bool isOpen() const;   int poll();   int poll(std::int64_t nowMs);   int work();
Persistence& persistence();   rune::client::Client& client();   const RuneStoreOptions& options() const;   RuneStoreStats stats() const;
bool close(std::chrono::milliseconds timeout);
std::size_t loadAll(Component component, const RecordFn& fn, std::chrono::milliseconds timeout);
std::string viewOf(std::string_view topic) const;
ItemMeaning
The three Store constructorsSee The rune broker. They are for a Game: the Store functions on a store made with a Registry throw std::logic_error, and the by-hand functions on one made like this.
RuneStore(registry, options)By hand: builds the client (rune's embedded driver starts, no connection). std::invalid_argument on bad options.
open()Connects, checks or creates the topics (a different fingerprint, or a missing topic without create, throws RuneStoreError), makes one producer per durability level in use and attaches the Persistence. std::logic_error if already open. After a throw the store is closed and open() may be called again.
poll(), poll(nowMs), work()Once per tick on the world thread, after registry.commit: work() (client work, then the outcome of each finished batch: acknowledge if every record completed Ok, else fail) and then persistence.collect. They return the work count and never block. poll() takes the time of the registry's last commit.
persistence()The Persistence (std::logic_error before open()).
stats()RuneStoreStats: batchesTaken, batchesRefused, batchesPartial, batchesAcknowledged, batchesFailed, recordsSent, recordsFailed, pumps, inFlightBatches, readsIssued, readsFloored, readsFound, readsAbsent, readsRetried, viewsReopened, readsFailed, readsInFlight, recordsReplayed (0 for the whole life of a game: nothing is replayed at start), and lastError.
close(timeout)Flushes everything, works until durable or the time is over, detaches the Persistence and closes the producers (the client stays connected). Returns whether it became durable; true at once if not open.
loadAll(component, fn, timeout)Blocking, for start-up: every record of the component's topic, latest value per key (tombstones left out). Returns the number of keys delivered. It sends an end marker, so the user needs WRITE (and READ on the topic). RuneStoreError on failure or timeout. each(topic, fn) is the same for tools and tests, O(topic); a Game never calls it.
RuneStoreErrorDerives std::runtime_error: what open, close and loadAll (and persist<Row>() in a Game) throw for a failure of the store or the broker.
createTopicStatement(topic, type), createViewStatement(topic, view, type)The rune QL an admin runs for a row; empty when the row has an enum member or no key.

ecs::persist, PersistOptions, Durability, OnRemove

PersistOptions persist(std::string topic);
template <class S, class T> PersistOptions persist(std::string topic, T S::* key);
enum class Durability : std::uint8_t { None, Leader, All };
enum class OnRemove : std::uint8_t { Keep, Tombstone };
// PersistOptions (each returns a modified copy)
PersistOptions conflate(std::chrono::milliseconds window);          // 1000 ms
PersistOptions durability(Durability d);                            // Leader
PersistOptions onRemove(OnRemove r);                                // Keep
PersistOptions staging(std::size_t rows);                           // 256
PersistOptions keyMember(std::string name);
template <class Saved, class ToSaved, class FromSaved> PersistWith<Saved> convert(ToSaved toSaved, FromSaved fromSaved);
PersistOptions view(std::string name);                              // "<topic>_by_<key member>"

registry.component<S>(options) registers a persisted struct; registry.component(type, options) a hand-built Type (it needs a .key() field; tags cannot be persisted). staging(rows) reserves records for rows that left memory and are not acknowledged yet, on top of the component's reserve; the area grows when full. The accessors topic(), keyMember(), conflate(), durability(), onRemove(), staging(), viewName(), viewGiven() and converter() read them back. The key must be an integer or a string of at most 16 bytes. convert needs a flat Saved (no List, no Entity) with a member named like the key: toSaved runs when a changed row is pushed, fromSaved when registry.load puts a record into a row.

ecs::Persistence

Persistence(Registry& registry, Sink sink, PersistenceOptions options = {});
void collect(std::int64_t nowMs);   void collect();
bool acknowledge(std::uint64_t sequence);   bool fail(std::uint64_t sequence);
Ticket flush(Entity entity);   Ticket flush();   bool durable(Ticket ticket) const;
std::optional<std::vector<std::uint8_t>> unsaved(Component component, std::span<const std::uint8_t> key) const;
std::uint64_t acked(Component component) const;
bool erase(Entity entity, Component component);
PersistenceStats stats() const;
using Sink = std::function<bool(const Batch&)>;
std::vector<PersistedComponent> persistedComponents(Registry& registry);
ItemMeaning
Persistence(registry, sink, options)Attaches to the registry (std::logic_error if it already has one); sink must not be empty. The destructor makes rows in flight dirty again, so nothing is lost for a later Persistence.
collect(nowMs), collect()Once per tick after registry.commit: finds the changed rows and pushes the due ones. nowMs is the caller's clock and must not go backwards. collect() takes the time of the last commit (std::logic_error if that commit had no time). Throws std::logic_error inside Query::each.
acknowledge(sequence), fail(sequence)The outcome of a batch handed to the sink. False if the sequence is not in flight (unknown, done, timed out).
flush(entity), flush()Pushes everything dirty of one entity (a logout) or of the registry (a shutdown), whatever the window, with the staged rows. Rows in flight are not pushed again until their outcome: call collect (or flush again) after it; the ticket covers those later pushes. A refused sink keeps the ticket waiting.
durable(ticket)True once every push the flush waited for is acknowledged; an empty flush is durable at once; it stays true. Tickets that are not asked about are dropped when more than 2 * kTicketsKept (256) wait, the newest kept; a dropped ticket answers true.
unsaved(component, key)The record of a key that left memory and whose last push is not acknowledged yet (an empty vector: a tombstone waits), or nothing. Game::load asks this first, so a player that leaves and joins again at once gets its last row. std::logic_error if not persisted.
acked(component)The sequence of the component's last acknowledged batch; 0 if none.
erase(entity, component)Writes a tombstone for the entity's row and removes the component. False if it has none.
persistedComponents(registry)The registry's persisted components, in registration order (component, topic, type, durability, fingerprint); needs no Persistence.

PersistenceOptions, PersistenceStats, PersistError, Batch

ItemDefault, meaning
maxBatchRecords512. A batch is cut when either limit is reached.
maxBatchBytes64 KiB. A single record larger than this goes alone.
inFlightTimeoutMs30000. 0: batches never time out.
onErrorOptional std::function<void(const PersistError&)>; must not throw. Kinds: KeyOverwritten (the key of a row was changed; it is restored; entity, component, expectedKey, foundKey), BatchFailed (fail(sequence) was called; component, sequence, rows), BatchTimedOut.
reserveBatches, reserveBatchBytes64 batches in flight reserved up front; 0 = maxBatchBytes for the batch buffer.
PersistenceStatsNow: dirtyRows, inFlightRows, inFlightBatches, stagedRows, ticketsHeld. Since the start: batchesSent, batchesAcknowledged, batchesFailed, batchesTimedOut, batchesRefused, rowsPushed, tombstonesPushed, keyErrors.
Batchtopic, fingerprint (of the component type), sequence (increasing per Persistence), durability, records ([keyLen u16][key][valueLen u32][value] ..., little-endian), count. Valid only during the sink call.

Not built here: pending updates, transactions, storedFirst.

See also: Server for the Game, Data for the three kinds of component, ECS for the registry, Testing.