ARCANUM
C++ client

Rune API

Connect, log in, define a typed topic, produce, subscribe, run a transaction, pick a delivery guarantee, have the broker check a write, query a view, hold a lease, grant access, handle errors.

Connect

One session to any node; leader redirects and reconnects are handled for you.

#include <rune/client/Client.h> using namespace rune::client; ClientConfig cfg; cfg.gatewayChannel = "aeron:udp?endpoint=node1:40200"; // any node cfg.gatewayStreamId = 1; cfg.egressChannel = "aeron:udp?endpoint=me:40400"; // where replies arrive cfg.egressStreamId = 2; cfg.username = "ws1"; // SCRAM-SHA-512 login cfg.passwordFile = "/run/secrets/ws1.pw"; // read once, never logged or sent cfg.offerCrypto = true; // TLS 1.3 cfg.requireCrypto = true; cfg.tlsExpectHost = "node1.example"; Client client(cfg); client.connectAndWait();

from your game loop

cfg.runnerMode = RunnerMode::Invoker; // no client threads // ... while (running) { client.doWork(); // once per server tick tickWorld(); }

Authentication

On by default. Each broker session logs in with SCRAM-SHA-512; the password never crosses the wire. A client without credentials is answered AuthenticationFailed.

ClientConfig cfg; cfg.username = "ws1"; cfg.passwordFile = "/run/secrets/ws1.pw"; // or cfg.password; never both // cfg.allowUnauthenticatedBroker = true; // accept a dev-mode broker (RUNE_AUTH=disabled) Client client(cfg); client.connectAndWait(); AuthStatus a = client.authStatus(); // Authenticated | PasswordChangeRequired | Anonymous

A client with credentials refuses a broker that skips the login, unless allowUnauthenticatedBroker is set. Revoking READ ends a live subscription with SubscriptionFailReason::NotAuthorized, a group consumer with GroupFailReason::NotAuthorized.

Topics

Defined in Rune QL from the rune console. A typed topic has at most one key field and validates every write; a record goes to the partition its key hashes to.

rune> create topic user { userId int64, name string(32), level int32 } partitions 16 key userId; rune> create topic character { characterId uint64 autoincrement, userId uint64, name string(16), role string(16), level int32 } partitions 1 replicas 3 key characterId with (cleanup.policy = compact); rune> create topic position { characterId int64, userId int64, zone int32, x float64, y float64, z float64 } partitions 1 key characterId; rune> alter topic character add column guildId int64, set compatibility backward;

replication, durability, retention

rune> create topic orders partitions 4 replicas 3 with (tier = leader, durability = all-in-sync); rune> create topic audit partitions 1 with (cleanup.policy = compact, segment.bytes = 1048576); rune> alter topic orders set retention.ms = 604800000, retention.bytes = 1073741824; rune> drop topic orders; rune> show history topic orders; -- every DDL is an event in the catalog

transient topic: replicated in memory, never written to disk. For high-rate, short-lived state.

rune> create transient topic presence { userId int64, online bool, zone int32 } partitions 16 key userId;

autoincrement: the broker assigns the key of every record written with key 0

rune> create topic ticket { ticketId uint64 autoincrement, userId uint64, subject string(64) } partitions 4 key ticketId with (cleanup.policy = compact, autoincrement.reserve = 4096, autoincrement.max = 4294967295);

The key is a uint64, int64 or uint32. It needs a new, persisted, compacted topic without retention limits, and it turns on key.index and modulo routing: a record goes to partition id mod partitions. The ids of partition p of P are n * P + p: dense per partition, never reused, with gaps after a failover, a rejected record or an aborted transaction. Past autoincrement.max a create is refused with 58 AutoIncrementExhausted and nothing is stored. The key field, its type and the partition count never change. Writing one: Checked writes.

import: write rows with the ids they already have

rune> grant import on topic 'ticket' to user backfill; rune> revoke import on topic 'ticket' from user backfill; -- when the import is done

A record with an explicit key is an import: an Insert stored as sent, after the counter is moved above it. It needs the IMPORT grant, which nothing implies, and a key at most autoincrement.max that belongs to its partition. A plain upsert is refused. Live creates that follow get fresh ids.

generate C++ types from the registry

rune schema generate character --out gen/ # gen/character.gen.h
or define a topic from code
auto d = rune::codec::DescriptorBuilder("user", 1) .add("userId", PrimType::Int64, 1, /*isKey=*/true) .add("level", PrimType::Int32) .build(); client.defineTopicAndWait("user", 16, "userId", /*validate=*/true, rune::codec::serialize(d));

Produce

Acks::None · Leader · All, plus the View flag (Checked writes). trySend is the non-blocking form.

auto p = client.producer({.acks = Acks::All, .lingerUs = 200}); TopicHandle positions = client.topic("position"); // or TopicHandle{topicId, partitionCount} p.send(positions, Key(characterId), bytes); // keyed, idempotent

many records: sends within lingerUs are batched automatically

for (const auto& e : events) p.send(positions, Key(e.characterId), e.bytes);

many records as one atomic append to one partition

auto b = p.batch(positions, Key(characterId)); b.add(step1); b.add(step2); b.add(step3); if (b.send() != OpResult::Accepted) { /* BackPressured: rebuild the whole batch */ } // contiguous offsets, all or nothing

On a topic with constraints the batch is also judged as one: a single rejected record rejects the whole batch (an atomic batch, kAttrAtomic). Sent with b.send(cq, token), it completes once with the code, the constraint id and the batch's base offset, and none of its records become visible.

zero-copy: write the record straight into the send buffer

auto c = p.tryClaim(positions, Key(characterId), PositionEncoder::kEncodedLength); PositionEncoder(c.buffer()).userId(userId).characterId(characterId).zone(4).x(118.5).y(7.0).z(-42.25); c.commit(); // or c.abort()

zero-copy batch

auto b = p.claimBatch(positions, Key(characterId), /*records=*/3, PositionEncoder::kEncodedLength); for (int i = 0; i < 3; ++i) PositionEncoder(b.record(i)).userId(userId).characterId(characterId).zone(4).x(path[i].x).y(path[i].y).z(path[i].z); b.commit(); // one atomic append

typed topic: generated flyweight, routed by the key

auto pos = client.openTopicAndWait<Position>("position"); auto w = pos.claim(characterId); w.zone(4).x(118.5).y(7.0).z(-42.25); w.commit();

Checked writes

The broker judges every write against the rules of the topic's views and key index. A rejected record is stored but never visible: no view folds it, no consumer receives it. Add Acks::View and the writer is told the verdict. Needs a broker with checked writes; an older one completes the send OpStatus::BrokerTooOld and nothing is judged.

typed writes: insert, update, upsert, remove, with a verdict

auto p = client.producer({.acks = Acks::All | Acks::View}); auto characters = client.openTopicAndWait<rune::gen::Character>("character", p); WriteResult r = characters.insertAndWait(rune::gen::Character{.userId = u, .name = "Ayla"}); // key 0: the broker assigns characterId; string(N) is rune::FixedString<N> if (r.ok()) return created(r.key()); // the new characterId if (r.code() == 55) { // ConstraintViolation auto c = client.constraintName(r.constraintId()); // {view, constraint} if (c && c->view == "character_name") return conflict(); // name taken if (c && c->constraint == "max_characters") return badRequest(); // 12 characters } WriteResult d = characters.removeExistingAndWait(id); // d.code() == 57 KeyNotFound: no such character
insert56 KeyExists when the key is stored. A key-0 create on an autoincrement topic is an insert
update · removeExisting57 KeyNotFound when the key is absent
upsert · removenever refused by the key index. A delete is never refused by a constraint either

Insert and update need a topic with a key index (with (key.index = on), implied by autoincrement) and an idempotent producer with View, or a transactional one. Plain upserts and deletes need neither. A local refusal throws std::invalid_argument from the …AndWait forms and returns SendResult::WriteNotAllowed from trySend.

what a rejection carries

r.code()55 ConstraintViolation from a unique or check view, 56, 57 from the key index; 0 when the write was accepted
r.constraintId()viewObjectId << 16 | ordinal: ordinal 0 is the view's unique, k its k-th constraint. 0 is the key index
r.offset()the rejected record's own offset
r.key()the id the broker assigned to a key-0 create; 0 for every other write and for an import
client.constraintName(id){view, constraint}, or std::nullopt for 0. A dropped or replaced view still resolves. A miss waits for one catalog refresh (at most 5 s): call it only from the thread that owns the Client

racing writers: the loser reads the winner's row from the unique view

rune> create topic login_user { userId uint64 autoincrement, provider uint16, subject string(64), createdAt int64 } partitions 1 replicas 3 key userId with (cleanup.policy = compact); rune> create unique view user_identity key (provider, subject) as select userId from login_user; // two first logins race for one (provider, subject); users: a TypedTopic<rune::gen::LoginUser> on a View producer WriteResult r = users.insertAndWait(u); if (r.code() == 55) { auto kb = v::ViewKeyBuilder::fromCatalog(state, "user_identity"); kb->setUInt("provider", provider).setString("subject", subject); ViewReadOptions ro{.minOffset = r.offset(), .partition = kb->partition()}; // wait until the view folded the loser's record ViewRowResult row = client.getViewRowAndWait(identityViewTopicId, *kb->key(), ro, 5s); // the winner's userId }

batches report every record

cq.poll([&](const Completion& c, std::span<const uint8_t> payload) { if (c.status != OpStatus::RecordsRejected) return; // Ok: every record was accepted BatchVerdicts v(c, payload); // c.value = the batch's base offset for (uint32_t k = 0; k < v.rejectedCount(); ++k) { RecordVerdict r = v.rejection(k); // r.index, r.errorCode, r.constraintId report(r.index, r.errorCode, client.constraintName(r.constraintId)); } // v.offsetOf(i): an accepted record's offset, -1 if rejected });

A claimBatch or raw Tier-2 batch of two or more records stores the accepted records and reports the rest. An atomic p.batch(...) is all or nothing and completes BrokerError once, with the constraint id and the base offset. A batch of one record keeps the single-record form. A View batch holds at most 4096 records.

autoincrement create

WriteResult r = tickets.insertAndWait(rune::gen::Ticket{.ticketId = 0, .userId = u, .subject = "bug"}); // key 0 uint64_t ticketId = r.key(); // the broker's id

A key-0 create needs an idempotent producer with Acks::All, plus View where a constraint can reject it. A producer never mixes key-0 creates with other records in one batch. A lost ack is retried and answers the original ids; a retry the broker can no longer match completes StoredIdUnknown: it was stored, its id is unknown. After an insertAndWait times out, look the row up through a unique view before inserting again. Explicit ids are imports: they answer key() 0 and need the IMPORT grant.

A retry inside the broker's window replays the original verdicts. Outside it the write completes OpStatus::VerdictUnknown: stored, verdict unknown.

Subscribe

Push delivery, in order per partition. key and value are zero-copy spans, valid inside the handler.

auto sub = client.subscribe(positions, Key(characterId), {.from = From::earliest(), .isolation = Isolation::ReadCommitted}, [](const Record& r) { apply(r.offset, r.key, r.value); });

Consumer group

Register before connectAndWait(). Offsets auto-commit; gc.commit() commits on demand.

GroupConfig g; g.autoCommitIntervalMs = 5000; auto gc = client.subscribeGroup("matchmaking", {"match_events"}, g); gc.onAssigned([](std::vector<TopicPartition> parts) { /* ... */ }); gc.onRecords ([](const RecordSet& rs) { /* rs.partition, rs.recordBatch */ }); client.connectAndWait();

Transaction

Any producer with a transactionalId. Every produce path works inside it: send, batch, tryClaim, claimBatch, typed claim. ReadCommitted readers see the trade only after commit.

auto p = client.producer({.acks = Acks::All, .transactionalId = "trade-worker-1"}); client.connectAndWait(); p.initTransactionsAndWait(); p.beginTransaction(); try { p.send(inventory, Key(int64_t{seller}), removeSword); p.send(inventory, Key(int64_t{buyer}), addSword); if (!canPay(buyer, price)) { p.abortTransactionAndWait(); // rollback: nothing becomes visible return; } auto c = p.tryClaim(wallets, Key(int64_t{seller}), WalletEncoder::kEncodedLength); WalletEncoder(c.buffer()).userId(seller).gold(price); c.commit(); // zero-copy, same transaction if (!p.commitTransactionAndWait().ok) return; // all or none; a no vote: already aborted (see below) } catch (const std::runtime_error&) { switch (p.state()) { case TxnState::CommittingTransaction: // timed out, outcome in doubt if (!p.commitTransactionAndWait().ok) return; // retry the same call break; case TxnState::FatalError: // fenced: a newer "trade-worker-1" took over shutdown(); break; default: // failed send, offsets or commit p.abortTransactionAndWait(); // rollback } }

constrained sources: a no vote is a result, not an exception

TxnCommitResult c = p.commitTransactionAndWait(); if (!c.ok) { // aborted: nothing in it is visible, the producer is Ready auto rule = client.constraintName(c.constraintId); // c.code: 55, 56, 57, 62 or 63 log(c.code, rule, c.failingOffset); // the first failing record p.beginTransaction(); // the next one }

At commit the leader of every partition the transaction wrote judges all its records together against every unique, check and key-index rule, and one no aborts the whole transaction. A commit that returns ok means the rows are final on every partition; View on a transactional producer is ignored. A retried commit after a coordinator failover gets the same no. Every other failure still throws. From a yes vote to the commit or abort marker the rows it wrote are locked: another writer's record on one of them waits for the marker. Deletes are never rejected. Within a partition a reader sees a transaction entirely or not at all.

Delivery guarantees

Chosen per producer and per consumer group.

at most once: no ack, no retry; commit before handling

auto p = client.producer({.acks = Acks::None}); p.send(positions, Key(characterId), bytes); GroupConfig g; g.autoCommit = false; auto gc = client.subscribeGroup("telemetry", {"position"}, g); gc.onRecords([&](const RecordSet& rs) { gc.commit(); handle(rs); });

at least once: retried until acked, deduplicated by the broker; commit after handling

auto p = client.producer({.acks = Acks::All}); p.send(positions, Key(characterId), bytes); GroupConfig g; g.autoCommit = false; auto gc = client.subscribeGroup("quests", {"character"}, g); gc.onRecords([&](const RecordSet& rs) { handle(rs); gc.commit(); });

exactly once: consume, produce and commit the input offsets in one transaction

GroupConfig g; g.autoCommit = false; g.isolation = IsolationLevel::ReadCommitted; auto gc = client.subscribeGroup("economy", {"trades"}, g); auto p = client.producer({.acks = Acks::All, .transactionalId = "economy-1"}); std::map<TopicPartitionKey, int64_t> offsets; std::vector<OffsetEntry> entries; gc.onRecords([&](const RecordSet& rs) { // Invoker mode: runs inside doWork() for (const auto& out : settle(rs)) p.send(wallets, Key(out.userId), out.bytes); offsets[{rs.topicId, rs.partition}] = rs.nextOffset(); }); client.connectAndWait(); p.initTransactionsAndWait(); p.beginTransaction(); while (running) { client.doWork(); if (offsets.empty()) continue; try { entries.clear(); for (const auto& [k, next] : offsets) entries.push_back({k.topicId, k.partition, next}); p.sendOffsetsToTransactionAndWait(entries, gc.groupMetadata()); if (!p.commitTransactionAndWait().ok) // outputs + offsets, atomically; a no vote aborted them gc.seekToCommittedAndWait(); // re-deliver the inputs } catch (const std::runtime_error&) { if (p.state() == TxnState::FatalError) return shutdown(); if (p.state() == TxnState::CommittingTransaction) { if (!p.commitTransactionAndWait().ok) // in doubt: retry the same call gc.seekToCommittedAndWait(); } else { p.abortTransactionAndWait(); // drop the outputs gc.seekToCommittedAndWait(); // re-deliver the inputs } } offsets.clear(); p.beginTransaction(); }

Materialized views

Defined in Rune QL, kept current on every write, read by key. Persisted by default: checkpointed and recovered on restart, no rebuild. Add transient to keep one in memory only. No separate database.

one row per character, keyed by account and character

rune> create view account_characters key (userId, characterId) as select name, role, level from character;

per-user stats over all of a user's characters

rune> create view user_stats key userId as select count(*) as characters, max(level) as top_level, avg(level) as avg_level from character;

rules the broker enforces: one character per name (any case), at most 12 characters per user

rune> create unique view character_name key lower(name) as nameKey as select characterId from character; rune> create view account_stats key userId as select count(*) as characters from character constraint max_characters check (characters <= 12);

A write that breaks a rule is refused with 55 ConstraintViolation and never becomes visible. A transaction that breaks one fails at commit with the same code and the constraint id, and nothing in it is applied.

unique projection viewno two source records may produce one view key. key lower(name) makes names unique in any case. A unique aggregate is refused (61)
constraint … check (…)a predicate over the view's own row (= <> < <= > >= against a constant, joined by and or not). A record is refused when its row fails it and the record changes a column the predicate reads. Projection or aggregate views
key indexthe Insert and Update intents, 56 and 57 (Checked writes)

A record that touches several constrained views must pass all of them, or changes none. Records in a batch are judged one at a time, in order, each seeing the accepted ones before it. Deletes are never rejected, so model a ledger as appended entries. A group already over its limit can shrink and never grow. NULL in a check passes. A unique view built over existing duplicates lets the smallest source key own the name; rune stats counts the other claimants as uniqueConflicts, and they can only shrink. Creating a constraint on a large topic pauses delivery to its consumers until the view has caught up; writes are never refused for it. create or replace keeps judging with the old definition until the new one has caught up. A constraint needs a persisted source (else 39), and at most 32 verdict views per topic (the key index counts as one), a view object id below 65536 and a plan record of at most 65,536 bytes (else 60).

wide and derived keys: up to 128 bytes of columns, lower() of a string

rune> create topic device_login { loginId uint64 autoincrement, userId uint64, deviceFp string(64), platform uint16, ts int64 } partitions 1 replicas 3 key loginId with (cleanup.policy = compact); rune> create view device_history key (userId, deviceFp) as select min(ts) as firstSeen, max(ts) as lastSeen, count(*) as logins, last(platform) as platform from device_login; rune> select characterId from character_name where nameKey = lower('Ada Lovelace');

A key component is an integer, bool or string(N) column; lower(name) as nameKey keys a string(N) by its lowercased value (Unicode simple lowercasing, the version pinned in the plan) and takes floor(3N / 2) bytes. A projection must stay identifiable: include the source key in the view key or declare it unique, else 59 ViewKeyExpressionInvalid. A view keyed by anything but the source key, and every aggregate view, needs a one-partition source (36). An aggregate group folds the latest record per source key, so to count events per user key the source by a per-event id (an autoincrement loginId), not by the user.

transient view: in memory only, rebuilt from its topics on restart. A view may read a keyed transient topic; a persisted one keeps its checkpointed rows across a restart.

rune> create transient view user_presence key userId as select count(*) as characters_online, last(zone) as last_zone from position;

query: a point read by the full key, or a scan by its leading columns

rune> select * from account_characters where userId = 42 and characterId = 7; name role level Ayla ranger 31 (1 rows) rune> select name, level from account_characters where userId = 42 limit 10; name level Ayla 31 Brin 24 (2 rows) rune> select name from account_characters where level = 31; error: where must name every key column (userId, characterId) for a point read, or the leading (userId) for a scan
where full keyone row
where userId = …the rows under that prefix; limit caps them
any other conditionrefused before anything is sent

from code: build the key, read a row or a prefix

auto kb = v::ViewKeyBuilder::fromCatalog(state, "account_characters"); kb->setUInt("userId", 42).setUInt("characterId", 7); ViewReadOptions ro{.partition = kb->partition()}; ViewRowResult row = client.getViewRowAndWait(viewTopicId, *kb->key(), ro, 5s); // point read kb->clear(); kb->setUInt("userId", 42); ViewScanResult page = client.scanViewAndWait(viewTopicId, *kb->prefix(), 0, ViewScanOptions{}, 5s); // every character of user 42

read your own write, and page through a prefix

ViewReadOptions ro{.minOffset = r.offset(), .partition = kb->partition()}; // waits until the view folded that offset ViewScanOptions so; for (;;) { ViewScanResult page = client.scanViewAndWait(viewTopicId, *kb->prefix(), 0, so, 5s); if (page.status == ViewRowResult::kCursorInvalidated) { so.resumeToken.clear(); continue; } // the prefix changed: start over if (page.status != ViewRowResult::kOk) break; for (const auto& row : page.rows) { /* decode like a point-read row */ } if (page.complete()) break; so.resumeToken = page.nextToken; }

ViewKeyBuilder encodes the key exactly as the broker folds it: setInt, setUInt, setBool, setString; key() needs every component, prefix() every one but the last, and lower() is applied for you. A page holds at most 256 rows and 61,440 row bytes; limit lowers the row cap. Pages come in table order, not key order. The reads need a broker with checked writes, else BrokerTooOld and nothing is sent. Status codes: kOk, kAbsentKey, kNotReady and kBusy (retry), kNotAuthorized, kCursorInvalidated.

A position update rewrites only that character's row. A user rename updates every row of that user. Deleting a character retracts its row and its contribution to user_stats.

inspect: definition, then the counters from each partition's leader

rune> show topic character; rune> show view account_stats; -- sources, key, outputs, unique, constraints, counters rune> create or replace view account_stats key userId as select count(*) as characters from character constraint max_characters check (characters <= 20); rune> drop view account_stats;
$ rune stats topic character --broker node1:40200 partition 0 rejectionsKeyExists 0 verdictLag 0 # records not yet judged autoIncrementCounter 1049 txnVotesNo 0 $ rune stats view account_stats --partition 0 --json --broker node1:40200 $ rune stats view character_name --broker node1:40200 constraint character_name.unique 3 # rejections, labelled view.constraint

show and rune stats need DESCRIBE on the object. rune stats (topic | view) NAME [--partition N] [--json] prints every partition's counters in id order (a counter the partition lacks is not printed), one line per constraint, and for a unique view uniqueConflicts. From code: client.getStatsAndWait(rpc::StatsScope::TopicPartition, topicId, partition, 5s) returns a StatsResult with get(StatId::verdictLag) and constraints; kBrokerTooOld means the seed broker predates it and nothing was sent.

Lease topics

A lease is a key held for a time to live and renewed by keepalives. When the holder stops, the key expires on its own. A lease topic is a typed, keyed topic whose rows carry four system columns the broker adds (lease_version, lease_ttl_ms, lease_holder, lease_remaining_ms). Every change to a lease is fenced by its version.

rune> create lease topic services { path string(128), uri string(128) } partitions 1 replicas 3 key path with (lease.ttl = 15000); rune> create transient lease topic online { userId uint64, zone int32 } partitions 4 key userId with (lease.ttl = 10000, lease.max_ttl = 60000);
lease.ttlthe ttl a grant gets when it names none. 30 000 ms
lease.min_ttl · lease.max_ttla shorter ttl is raised to the floor (at least 1000 ms); a longer one is refused (66). Up to 9 000 000 000 000 ms. A transient lease topic sets lease.max_ttl explicitly
lease.max_keyslive leases per partition, 1 to 16 777 216, creation-only. Past it a grant is refused with 65
lease.expiry_batch · lease.checkpointexpiry records per batch (1 to 4096); checkpoints (on by default) that record the remaining time of long leases. Also lease.checkpoint_interval and lease.checkpoint_rate

The CLI parses show leases, show lease and revoke lease but does not run them yet; read and revoke leases from code.

grant, keep alive, revoke

auto services = client.openLeaseTopicAndWait<rune::gen::Services>("services"); rune::gen::Services row{}; // path, uri: rune::FixedString<128> row.path = "eu/matchmaker/1"; LeaseResult g = services.grantAndWait(row, LeaseOptions{.ttlMs = 15000, .holder = nodeId}); if (!g.ok()) { /* BrokerError: g.errorCode 56 KeyExists, 65 LeaseLimitExceeded; a ttl above lease.max_ttl is refused locally (InvalidArgument) */ } // g.version fences every later call; g.ttlMs is the ttl granted LeaseResult k = services.keepAliveOnceAndWait(row, g.version); // k.errorCode 64 LeaseNotFound: expired, revoked, or another version LeaseResult u = services.updateAndWait(row, g.version); // compare-and-set on the version; does not renew LeaseTtlInfo info; LeaseResult t = services.timeToLiveAndWait(row, &info); // info.remainingMs, info.version, info.holder LeaseResult v = services.revokeAndWait(row, g.version); // check each result: !ok() means the call did not apply

keep leases alive from the game loop

auto onLost = [&](LeaseRef ref, LostReason why) { stepDown(); }; auto onAtRisk = [&](LeaseRef ref, AtRiskEdge edge) { /* Entered: stop acting as holder; Cleared: resume */ }; LeaseSession<64, decltype(onLost), decltype(onAtRisk)> session(client, onLost, onAtRisk); int64_t grantSentNs = client.nowNs(); // taken before the grant is sent LeaseResult g = services.grantAndWait(row, LeaseOptions{.ttlMs = 15000}); LeaseRef ref = session.add(services, row, g.version, /*ttlMs=*/15000, grantSentNs); while (session.bindProducer(ref, writer) == OpResult::BackPressured) session.doWork(client.nowNs()); // before the writer's first write; its writes fail when the lease is lost or at risk while (running) { session.doWork(client.nowNs()); // keepalives every ttl/3; fires onLost(ref, reason) / onAtRisk(ref, edge) tickWorld(); }

A session sends keepalives at a third of the ttl. A lease that was not renewed in time, or that the broker no longer has, is lost: its bound producers fail first (DeliveryFailed), then onLost fires once and a fenced revoke goes out. A lease that is only at risk fails the unacknowledged and new writes of its producers the same way (DeliveryFailed, LeaseAtRisk) and fires onAtRisk(ref, Entered); after the next answered keepalive (Cleared) their writes go out again. Grants, updates, revokes and keepalives need WRITE on the topic; timeToLive and reads need READ. A persisted lease topic survives a restart; a transient one forgets every lease when all replicas of a partition restart, its watchers get 43 PartitionDataReset and holders grant again. Needs a broker with lease support (an older one: openLeaseTopicAndWait throws BrokerTooOld); a session below client version 18 is answered 39.

Users, roles, grants

Rune QL statements, run as an admin with rune ql -f. Explicit allow, default deny. Every change is an event in the replayable __security log.

first admin: once per cluster, proves possession of the cluster key

$ rune security gen-cluster-key --out secrets/cluster.key $ rune security bootstrap --admin admin --cluster-key-file secrets/cluster.key --broker node1:40200 one-time password for admin: ... # printed once; change it at first login

users.rql, run with rune ql --user admin --password-file admin.pw -f users.rql --broker node1:40200

CREATE ROLE world_server; GRANT READ ON TOPIC PREFIX 'world_' TO ROLE world_server; GRANT WRITE ON TOPIC PREFIX 'world_' TO ROLE world_server; GRANT READ ON GROUP PREFIX 'world-' TO ROLE world_server; GRANT WRITE ON TRANSACTIONAL_ID 'ws-1' TO USER ws1; CREATE USER ws1 WITH PASSWORD 'correct horse battery'; GRANT ROLE world_server TO USER ws1; ALTER USER ws1 ADD PASSWORD 'a second password 1234'; -- rotate: add, roll clients, then ALTER USER ws1 DROP PASSWORD 1; ALTER USER ws1 DISABLE; REVOKE WRITE ON TOPIC PREFIX 'world_' FROM ROLE world_server; SHOW GRANTS FOR USER ws1; SHOW SECURITY HISTORY USER ws1;
operationsREAD WRITE CREATE ALTER DROP DESCRIBE ADMIN IMPORT
resourcesCLUSTER TOPIC VIEW GROUP TRANSACTIONAL_ID
matchan exact name, or PREFIX 'p'
passwords12 characters or more; at most two active; read from a file or stdin, never -e

from code

auto stmt = rune::ql::parse("GRANT READ ON TOPIC PREFIX 'world_' TO ROLE world_server;"); auto plan = rune::ql::applySecurityPlan(*stmt); SecurityDdlResult r = client.securityDdlAndWait(std::move(*plan), DdlOptions{}, std::chrono::seconds(15)); auto users = client.securityUsers(std::chrono::seconds(10)); // ADMIN on CLUSTER

Revoking READ ends the affected subscriptions with NotAuthorized; revoking WRITE fails the next produce. Dropping or disabling a user, or TERMINATE SESSIONS, answers every live session SessionRevoked. A permission denied on an object you cannot see looks the same as a missing object.

Errors

*AndWait throws std::runtime_error (timed out, session lost, broker error code 45 (NotAuthorized): ...). A broker error on a DDL or schema call is returned in the result: errorCode and detail. The token form completes with an OpStatus and the code.

2 NotLeaderretried after a metadata refresh; 3 Redirect is followed
6 Internala DDL answered 6 is in doubt: it may still apply. Retry with the same request id
7 UnknownTopic · 8 UnknownPartition
10 MinIsrViolationacks=all with too few in-sync replicas: retried after a backoff
11–16consumer group: StaleGeneration, UnknownMember, RebalanceInProgress, NotCoordinator, IllegalGeneration, GroupDead
17–19transactions: InvalidProducerEpoch, ProducerFenced, InvalidTxnState
20 OutOfOrderSequence · 21 DuplicateSequenceidempotent produce; a known duplicate counts as success
22 MessageTooLargea batch above the broker's limit; a record above 16 KiB is rejected at append
23 InvalidSchema · 24 SchemaValidationFailedbad descriptor or partition count; a record failed validation
25 TopicExists · 40 InvalidTopicNamename taken (case-insensitively); bad name
26 StorageErrora log failed an fsync; the broker halts
27 ConcurrentTransactions · 28 UnknownProducerIdretried; re-sequenced
29 HandshakeFailedTLS could not be negotiated, or RUNE_REQUIRE_TLS refused a plaintext client
30 SchemaIncompatible · 31 SchemaConflicta change the compatibility mode refuses; a stale expectedVersion
32–38views: ViewExists, ViewSourceNotFound, ViewFieldUnresolved, ViewKeyInvalid, ViewNotCoPartitioned, ViewSourceTransient, ViewNotAlterable
39 InvalidRequesta malformed request or an out-of-range DDL setting; never retried
41 ViewSourceIsViewa view cannot read another view
42 TransientLogFulla transient partition's ring is full of unreplicated records: retried after 100 ms
43 PartitionDataResetthe transient data a subscription resumed from is gone; it restarts at the log start
44–47authentication: AuthenticationFailed, NotAuthorized, SessionRevoked, PasswordChangeRequired
48 AuthUnavailablethe broker's security state is not current: back off and reconnect, rotating brokers
49 RecordEgressUnavailable · 50 UnstableOffsetCommit49: retried a few times, then fails the subscription or group consumer. 50: retried after a backoff while a transaction holds the offset, never surfaced
51–54namespaces: NamespaceSuspended, NamespaceLimitExceeded, Throttled (carries a retry-after), ViewPaused
55–57a write broke a view rule: ConstraintViolation, KeyExists, KeyNotFound. Never visible; the constraint id names the rule and the offset the record. In WriteResult and BatchVerdicts; in a transaction, in TxnCommitResult
58 AutoIncrementExhaustedthe counter reached its maximum; nothing is stored
59–61view definition errors: ViewKeyExpressionInvalid, ViewConstraintInvalid, ViewUniqueInvalid
62 TxnTooLarge · 63 TxnConflicta transaction voted no at commit; it is already aborted: begin the next one
64 LeaseNotFoundno live lease for the key, or at another version
65 LeaseLimitExceeded · 66 LeaseTtlTooLargelease.max_keys or the namespace lease limit reached; a ttl above lease.max_ttl
codes 55–66sent only to sessions at the matching client version; an older client is answered 39 InvalidRequest
OpStatus::BrokerTooOld · kBrokerTooOldthe seed broker predates checked writes, stats or leases: nothing was sent, and nothing was judged. Upgrade the broker
OpStatus::VerdictUnknown · StoredIdUnknowna retry fell outside the broker's window: the write was stored, its verdict or its autoincrement id is unknown. Look it up through a view
OpStatus::RecordsRejecteda batch of two or more records had rejections; BatchVerdicts lists them