Hanamori on Rune, part 1: topics, views and boot
How Hanamori's meadow and realm servers share everything through 14 Rune topics and 7 views: the vocabulary, how a server boots, and how a player logs in. With the real code.
Hanamori is a game with a big shared field and a lot of small private dungeons. The field is run by one meadow server. The dungeons are run by any number of realm servers. This post is part 1 of 3 about how they cooperate, and the short version is that they do not. They never open a connection to each other. Everything they need to tell each other goes through Rune, a log-based message broker, as records on named topics. Inside one server, threads talk through rings; that story is in the previous post.
The series: part 1 (this one) covers the vocabulary, the 14 topics, the 7 views, how a server starts up and how a player logs in. Part 2 covers the things that change all the time: positions, XP, gold, inventory, loot and portals. Part 3 follows one hero from the meadow into a dungeon and back, including everything that can go wrong on the way.
Each idea below is explained, then shown in code and in a diagram. The code is trimmed, never invented: the names are the real types and functions.
1. Servers talk through Rune, not to each other
Rune is the broker. The meadow server and the realm servers are its clients, and what they exchange are records. Records travel over Aeron, a UDP transport, which is a polite way of saying that packets get lost and somebody has to notice.
2. What a topic is
A topic is an append-only log of records. Each record has a key (which thing it is about, such as a character id), a
value (a typed row with a schema), and an offset (its line number). Hanamori declares every topic once, in a table called kTopics, and both
servers use the same table. Each row says how many partitions the topic has, how old records are cleaned up, how durable a write must be, and which
field is the key.
// meadow-server: Topology.h (shape of one row, trimmed) struct TopicSpec { const char* name; // "meadow_positions" int partitions;// 4 Cleanup cleanup; // compact, or delete (a ledger) Acks acks; // Leader (fast) or All (durable) Persistence persist; // Disk, or Transient (memory only) TopicRole role; // BootReplay, TailOnly or WriteOnly const char* key; // field used as the record key };
Three choices in that row matter for everything that follows.
Compact or ledger
A compact topic keeps the latest record per key, so the topic keeps only current values: old values for the same key are
eventually discarded. A delete topic with retention -1 keeps everything forever, so it is a ledger. Hanamori's two ledgers are
meadow_xp and meadow_gold. Their key is a random per-entry id, never the character, so two rewards can never overwrite each other, and the
character's total is the sum of the entries. A compact record with an empty value is a tombstone: it means "this key no longer exists", which is how a
picked-up item or a closed portal disappears.
Durable or fast
Acks::Leader returns as soon as the topic's leader has the record. Acks::All waits until the replicas have it too. Positions
and spawn-point deadlines use Leader, because losing the last second of movement is cheaper than making every step wait for a quorum. Everything
that represents something a player would complain about, such as gold, items and who owns a character, uses All.
Disk or transient
Persistence::Disk survives a broker restart. Persistence::Transient is replicated in memory and is empty after a broker restart. The four
realm_* topics are transient: they carry heartbeats, requests, events and tickets, all of which are worthless a minute later. The network, as usual, is welcome to lose
them; the design assumes it will.
3. The 14 topics
Nine topics hold the shared world, one holds who currently owns each character, and four carry realm coordination. The last column is the
role: BootReplay topics are read from the beginning at startup, TailOnly topics are only followed live, and
WriteOnly topics are never read by the meadow.
| Topic | Partitions · kind | Key | Meaning |
|---|---|---|---|
meadow_characters | 1 · compact, All, Disk | id | name, secret hash, optional password hash |
meadow_positions | 4 · compact, Leader, Disk | id | last known position |
meadow_attributes | 1 · compact, All, Disk | id | spent attribute points |
meadow_xp | 1 · ledger, All, Disk | entryId | XP rewards |
meadow_gold | 1 · ledger, All, Disk | entryId | gold in and out |
meadow_inventory | 1 · compact, All, Disk | slotKey | one record per bag or equipment slot |
meadow_portals | 1 · compact, All, Disk | portalId | open portals |
meadow_ground_items | 1 · compact, All, Disk | itemId | loot lying on the ground |
meadow_spawn_points | 1 · compact, Leader, Disk | spawnPoint | respawn deadline per spawn point |
character_location | 1 · compact, All, Disk | characterId | who owns the character: meadow or a realm |
realm_servers | 1 · compact, All, Transient | serverId | realm heartbeat |
realm_requests | 1 · delete, All, Transient | requestId | meadow asks a realm to create or cancel an instance |
realm_events | 1 · delete, All, Transient | instanceId | realm reports created, rejected, joined, left, ended, cleared |
realm_tickets | 1 · compact, All, Transient | characterId | the one-time entry pass for a hero |
Ten topics are BootReplay: all nine meadow_* topics plus character_location. realm_servers and
realm_events are TailOnly, and realm_requests and realm_tickets are WriteOnly for the meadow.
Note on keys: key 0 is reserved for boot markers (section 6), so character ids start at 1. An inventory slot key packs three facts into one
number, so each slot is one record that can be overwritten:
// Schemas.h // (characterId << 16) | (container << 8) | idx // container 0 = bag: idx = y*10 + x // container 1 = equipment: idx = slot index uint64_t inventorySlotKey(characterId, container, x, y);
4. Views: running totals maintained by the broker
A Rune view is a table maintained by the broker from one source topic. Each output field has a merge rule: Replace keeps the
latest value, Sum adds. The meadow defines seven of them at startup in defineViews_.
// shape of a view plan (Rune client API); meadow_xp_total as the meadow defines it
v::ViewPlan plan;
plan.viewName = "meadow_xp_total";
plan.key = "characterId";
plan.sources.push_back(v::SourceSpec{xpTopicId, "characterId"});
plan.output.push_back({"characterId", PrimType::UInt64, 0, "characterId", v::MergeRule::Replace});
plan.output.push_back({"total", PrimType::Int64, 0, "delta", v::MergeRule::Sum});
| View | Rule | Source |
|---|---|---|
meadow_character | Replace (secret and password hashes left out) | meadow_characters |
meadow_position | Replace | meadow_positions |
meadow_character_attributes | Replace | meadow_attributes |
meadow_xp_total | Sum of delta, field total | meadow_xp |
meadow_gold_balance | Sum of delta | meadow_gold |
character_inventory | Replace, keyed by slot | meadow_inventory |
meadow_portals_live | Replace | meadow_portals |
An honest footnote: no game server reads these views. The servers rebuild their state by replaying the topics. The views are there for everything else, such as dashboards and tools, and they leave the secret and password hashes out on purpose. Two Rune rules shaped the design: an aggregate view needs a one-partition source, which is why the ledgers have one partition, and a view over a transient topic must itself be transient.
5. One thread owns the connection
A single thread, RuneProcessor, owns the Rune client (in Invoker mode, running on an AgentRunner). The
Account, Sim and Network threads never touch the broker. They send messages into the Rune thread's inbox, a many-to-one ring, and receive records back through
their own inboxes. Records for characters go to Account. Everything else goes to Sim (routeRecord). The realm server has the same shape with its own
processor.
6. Booting: how a server finds out it is caught up
At startup RuneProcessor::start_() does this, in order:
- defines the topics and views (an "already exists" error, codes 25 and 32, is fine; transient broker errors 48 and 49 are retried up to 6 times);
- initialises a transactional producer named
meadow-world; - subscribes with
From::earliest()andIsolation::ReadCommittedto every topic that is not write-only (12 of 14); - sends one marker (key
0, a random non-zero nonce) to every partition of everyBootReplaytopic: 13 partitions across 10 topics.
The class BootReplay counts partitions. When a partition delivers its own marker, that partition is caught up. A key-0 record with a different nonce
is a stale marker from an earlier run and is dropped. When every partition has answered, done() is true and finishBoot_() sends
BOOT_DONE to Account, Sim and Network. Markers on compacted topics are then tombstoned so they do not pile up; ledger markers have a delta of 0 and simply stay.
If a marker goes missing, resendMarkers_ sends them again every 5 s.
// BootReplay.h (shape) void onRecord(int topicIdx, int partition, uint64_t key, uint64_t stamp) { if (key != 0) return; // real data: not our business if (stamp != nonce_) return; // a marker from some earlier boot caughtUp_[topicIdx][partition] = true; // this partition has reached "now" } bool done() const; // every partition of every boot topic
While boot is running, the rest of the server behaves. The Account thread answers any join with TRY_AGAIN (the booted_ flag is false). The Sim thread
defers ADMIT and CONN_CLOSED until BOOT_DONE, then replays them in order (replayDeferred). The health endpoint reports ok only after
bootDone. If a boot subscription ends, there is nothing sensible to do and the process exits with code 1; the orchestrator can start it again.
After boot, the Rune thread stops forwarding most history. Its tail filters (tailTopicAccepts, tailWriterAccepts) let through only the topics that
other writers also touch: inventory, gold, XP and attributes, and only records written by a realm (writerId non-zero). Tombstones are dropped, and the meadow's own
writes (writerId 0) are skipped, because it already knows what it wrote. The three realm topics are always forwarded in full. Positions, portals, ground items and
spawn points are not tailed at all after boot: the meadow is their only writer. One trap is worth naming: a record with quantity 0 on meadow_inventory is not a tombstone.
The realm writes a cleared slot as quantity 0, and that record must be delivered.
7. Logging in
The Account thread (AccountCore) handles the whole join. The slow work, hashing passwords with Argon2id, lives on this thread so the world tick never waits for it.
A new name
If the name is free, the Account thread creates a character with WriteCharacter. The server mints a random secret for the browser to keep and stores only its SHA-256 hash (64 hex characters). A new name with no credential gets the secret handed back; a new name with a password of at least 8 characters (MIN_PASSWORD) claims the name with that password.
The id is the highest id so far plus one. The write is a transaction with five records: the character, plus four starter items from
STARTER_GEAR (a worn sword, buckler, apprentice wand and padded jerkin, in equipment slots 0, 1, 2 and 4). All five land or none do, so there is no hero without a sword,
which in a fantasy game is a legal minimum. If the commit fails, the retry mints a new character id rather than reusing the failed one.
A returning name
The Account thread checks the credential in constant time: ctEqual(sha256(cred), secretHash) for the secret, or argon2idVerify for the password. On success it writes
lastLoginAt without waiting. A pending password change defers that write so the two do not race. Failures come back as a JoinFailCode:
| Code | Value | When |
|---|---|---|
| NAME_TAKEN | 1 | the name exists and the request carried no credential, or the name is being created right now |
| BAD_CREDENTIALS | 2 | wrong secret or password |
| NAME_INVALID | 3 | the name is not acceptable |
| RATE_LIMITED | 4 | JoinLimiter: too many attempts per address or per name |
| SERVER_FULL | 5 | no room |
| TRY_AGAIN | 6 | booting, or a transient write failure |
| BAD_VERSION | 7 | the client is too old |
Password changes are separately limited to 3 per character per minute (MAX_SET_PASSWORD_PER_MIN).
Finally Sim places the player: at the last known position from lastPos_ if it is finite and inside the map, otherwise at the default spawn. A first-time player also triggers
onFirstJoin, which opens a portal 13 m ahead, so nobody has to wonder where the content is.
8. How writes leave: the transaction queue
Everything above that says "a transaction" goes through TxnQueue on the Rune thread. It keeps these promises:
- One transaction is open at a time, first in, first out. A message from the ring (
msg::Txn) holds at most 8 operations (MAX_OPS). - Exactly one reply per job. Each submitted job gets exactly one
popDone: on success, failure, timeout, refusal, being fenced, or shutdown. Callers never wait on silence. - Before the commit is issued, a transaction that runs longer than 5 s is given up on. After commit is issued the outcome comes back as
Ok,Definitive(it failed; abort) orInDoubt(the network is being mysterious). An in-doubt commit re-issues the same commit, for up to 60 s. Rune refuses to abort a transaction that is already committing, so a commit that timed out may still land. The code therefore never assumes a timeout means "no". - Being fenced (another producer took over the same transactional id) is fatal. The process exits with code 3.
// TxnQueue.h (shape) enum class CommitResult { Ok, Definitive, InDoubt }; // InDoubt -> commitTransaction() again, same transaction, never a new one // Definitive -> abortTransaction(), job completes with failure // Fenced -> kExitFenced (3)
That is the vocabulary. Part 2 uses it to follow the data that changes every second: where players stand, what they earn, what lies on the ground and when portals open.