ARCANUM
Guide ยท Data

Entities, components and the contract

Everything a game stores is a component on an entity. This page builds the data model one idea at a time, then shows the three kinds of component, how to read and write them on the server, and the contract file that keeps a server and its clients in agreement.

Entities

An entity is only an id. It carries no data and no behaviour: it is the thing that components are attached to. In meadow, every player is one entity, and so is every sheep and every coin.

The id is a 64-bit value with two halves: the low 32 bits are a slot, an index into the registry's tables, and the high 32 bits are the slot's generation. When an entity is destroyed its slot may be reused, but with the generation raised by one, so the id of a destroyed entity never matches a later entity that took its place.

entities.cpp
#include <aether/ecs.h>
#include <print>

using namespace aether;

int main() {
    ecs::Registry registry;

    ecs::Entity a = registry.create();
    std::println("{}", a);                         // 0:1   slot 0, generation 1
    registry.destroy(a);

    ecs::Entity b = registry.create();             // takes slot 0 again, with generation 2
    std::println("{} {} {}", b, registry.alive(a), registry.alive(b));   // 0:2 false true
}

Notice that the first entity is 0:1, not 0:0: generation 0 is never alive, so an Entity that was never set (all zero bits) can never name a real entity by accident.

Is it set, and is it alive?

These are two different questions, and each has its own function.

  • ecs::isSet(e) looks only at the id. It is false for ecs::kNoEntity (all bits set, the value used for "no entity") and for ecs::Entity{} (zero, what an Entity member of a row holds until someone sets it). It says nothing about the world.
  • registry.alive(e) asks the registry whether the entity still exists. In a server::Game the registry is registry().

Printing an entity gives slot:generation, and an unset entity prints none. This is the form every error message uses. static_cast<std::uint64_t>(e) gives the raw id; Entity is an enum class on purpose, so there is no arithmetic and no implicit conversion to a number or to bool.

Trap. An entity id means nothing after the server restarts, and it cannot be stored in the database (see Saving). Whatever must survive a restart is identified by a key that you choose (a name, a number), not by an entity id.

Using the id of a destroyed entity is a mistake that is reported rather than ignored. Adding a component to it throws std::logic_error:

ecs::Registry::add: entity 0:1 is dead (it was destroyed, or it is from another registry): use an entity from create() that has not been destroyed

With a server::Game you rarely call create() yourself: spawn(...) makes an entity and gives it its first components in one step (see Reading and writing rows on the server).

Components, rows, fields and types

Data lives in components. A component is a named kind of data that an entity may have or not have: Walker (a position and a speed), Purse (some gold). An entity has at most one of each component.

The registry keeps one table per component. The data of one component on one entity is a row of that table. A row is made of fields: fixed-size leaves such as one double, one 16-byte string or one flag. The description of those fields (their names, their order, their kinds and widths) is the component's type, an ecs::Type.

meadow's Walker is the smallest useful example:

examples/meadow/walker_type.cpp
#include <aether/ecs.h>
#include <print>
#include <meadow/Messages.h>      // generated: struct meadow::Walker { double x, y, speed; }

using namespace aether;

int main() {
    ecs::Registry registry;
    auto walker = registry.component<meadow::Walker>();
    const ecs::Type& type = ecs::Component(walker).type();
    for (const auto& f : type.fields()) std::println("{} offset {} width {}", f.name, f.offset, f.width);
    std::println("one row is {} bytes", type.blockLength());
}
x offset 0 width 8
y offset 8 width 8
speed offset 16 width 8
one row is 24 bytes

Notice what the type is for. A row has a fixed size, known from its type alone. That is what lets the registry keep rows in a flat table, compare them byte by byte, copy them cheaply, and describe them to a client that was not compiled against your structs. Everything that follows (what a member may be, why a std::map cannot be a row) comes from that one rule.

Three words you will see everywhere. Component: the kind of data. Row: that data on one entity. Field: one fixed-size leaf of a row. A struct with a nested struct and an array flattens to many fields; a field name such as position.x or hotbar[3] names one of them.

The next section shows that you can define a component in three different ways. After that, each way gets its own section.

The three kinds of component

Most of what a game stores is simple data: positions, health, names. A little of it is not: a map of paths, a cache, a pointer to a shared mesh. Aether has a home for each.

  1. A flat struct component. You write a plain C++ struct with fixed-size members. Aether finds its members by reflection, stores the struct as it is, and can send it to clients and save it to a database. This is the normal case and needs no macro.
  2. A hand-built ecs::Type component. You describe the fields at run time with a builder, and rows are packed bytes read through Field<T> handles. Use it when there is no C++ struct to write: data-driven definitions, tags, a scripting layer.
  3. A local component. Any movable C++ type: a std::map, a std::vector, a std::shared_ptr, a class with constructors. It lives on the server only. It is never sent and never saved by the registry; you show part of it to clients by writing a function that fills a plain row.

The first two are flat: a fixed block of fixed-size fields. They behave the same once registered (both can be sent through a view, both can be saved); they differ only in how you define them. The third is a different thing: a live C++ object.

1. Flat struct2. Hand-built Type3. Local
You writea plain structType::define("name")...build()any movable C++ type
Registered withregistry.component<S>(), or implicitly by view<S>() / spawn(S{})registry.component(type)registry.local<T>(), Game::local<T>(), or implicitly by spawn(T{}) for a non-flat T
Handle you getTyped<S> (converts to Component)ComponentLocal<T> (converts to Component)
You read it asthe struct, S&Row / ConstRow and Field<T>the object, T&
In memorythe struct itself (sizeof(S), padding included)packed bytes, no paddingthe live object, moved by the pool
Member / field typesfixed-size only (see the list)the Prim kinds (list)anything
Requirementsplain aggregate, trivially copyable, alignment at most 64none beyond the builder's own rules (a type with no fields is a tag)movable, noexcept move constructor and destructor, alignment at most 64
A source of a view (from, join, select, groupBy)yesyes (not a tag)no: a compile error. Only view.from(local, function)
Sent to clientsthrough a viewthrough a viewnever directly; only what a function view copies out
Savedyes, if flat enough (Saving)yes, if it has a key() fieldnever by the registry
Queries, hooks, deferred changesyesyesyes
Survives a restartif savedif savedno
Which one? If clients or the database must see it: a plain struct. If it must be built at run time or has no fields: a Type. If it is a server-side object that no client reads as it is: a local component, plus a function view for the part that clients should see.

Kind 1: the flat struct component

The simplest component is a struct. Suppose meadow gets sheep. A sheep has a name, a position, a colour, a short trail of recent places and a mood:

examples/meadow/Sheep.h
#pragma once
#include <array>
#include <cstdint>

#include <aether/shared.h>

namespace meadow {

using namespace aether;

enum class Mood : std::uint8_t { Calm, Grazing, Angry };

struct Vec2 { double x, y; };

struct Sheep {
    String<16> name;
    Vec2 position;
    std::array<std::uint8_t, 3> colour;     // red, green, blue
    List<Vec2, 4> trail;                    // the last four places it stood
    Mood mood;
    bool shorn;
};

}  // namespace meadow

That is all. There is no macro and no list of members. Aether reads the struct's members, their types and their names at compile time, by the technique of Boost.PFR (the compiler's pretty-function name gives the names). The SDK is built and tested as C++23; the reflection header itself needs C++20 and has been checked with GCC 15.2 and Clang 22. The component is called Sheep: the struct's name without its namespace. The row is the members in declaration order.

You register it, add it to an entity and read it back. At registry level that is:

examples/meadow/sheep_registry.cpp
#include <aether/ecs.h>
#include <print>

#include "meadow/Sheep.h"

using namespace aether;
using meadow::Sheep;

int main() {
    ecs::Registry registry;
    auto sheep = registry.component<Sheep>();          // Typed<Sheep>

    ecs::Entity e = registry.create();
    Sheep s{};
    s.name = "Dolly";                                    // a String<16>
    s.position = {10, 20};
    s.trail.push({10, 19});
    registry.add(e, sheep, s);                           // a row with these values

    Sheep& row = registry.get(e, sheep);                 // the row itself, as the struct
    row.position.x += 1;                                 // changes it in place: nothing to write back
    std::println("{} at {}, {}", row.name.view(), row.position.x, row.position.y);
}

Notice that registry.get(e, sheep) hands back Sheep&, a reference into the table. Because a flat component is stored as the struct, there is nothing to copy out and nothing to write back.

What a member may be

A row is a fixed block, so every member must have a fixed size. This is the complete list.

Member typeBecomesNotes
std::int8_t, int16_t, int32_t, int64_t, std::uint8_t, uint16_t, uint32_t, uint64_tone integer field of that widthAny spelling of those widths is fine (int, long long, unsigned). Not bool, and not a character type (char, wchar_t, char8_t, char16_t, char32_t).
an enum with a fixed underlying typethe integer of that typeenum class Mood : std::uint8_t and enum Colour : int are fine. An unscoped enum with no : type is refused: the compiler picks its underlying type, and it is not the same on every compiler.
float, doublea 4-byte or 8-byte float field
boola 1-byte flag
String<N>a field of N bytes, zero paddedN from 1 to 65535. See Strings.
ecs::Entitya uint64 fieldAllowed in a row that is sent. Not allowed in a row that is saved (Saving).
a plain struct of allowed membersits members, in place, named outer.innerNested to any depth. It cannot contain itself.
std::array<T, N> of an allowed typeN fields, name[0] to name[N-1]Arrays of arrays are fine.
List<T, N> of an allowed typea count field, then N slotsN from 1 to 65535. The item is not itself a List. See Lists.

aether::String and aether::List are the same types as ecs::String and ecs::List.

Strings

String<N> owns N bytes. The text is what comes before the first zero byte, so it holds at most N bytes of text. It is copied with the row, so it is safe to keep; std::string_view would not be (it points at memory it does not own), which is why a view of a string is not a member type.

String<16> name{"Ayla"};                 // explicit construction from text
name = "Brin";                           // assignment from std::string_view, std::string or a literal
std::string_view v = name;               // converts to string_view
name.size();  name.empty();  name.str(); // 4, false, a std::string copy
name == "Brin";                          // compares the text

A string that does not fit is never cut short. It throws std::invalid_argument and leaves the value as it was:

ecs::String<4>: the string has 7 bytes and does not fit (it is never truncated). Cut it before, or use a larger String
Trap. In a handler this exception loses that one message; in onJoin it refuses the client. Cut user text to the size you promised before you store it: String<16>(name.substr(0, 16)). Remember that N counts bytes, not characters: a UTF-8 name of 16 characters may need more than 16 bytes.

Lists

A row has one size, so a collection inside a row must have a limit. List<T, N> is a count and room for N items. It never grows and never drops an item.

List<Vec2, 4> trail;
trail.push({1, 2});          // adds at the end
trail.erase(0);              // removes item 0; the later ones move up (order kept)
trail.size();  trail.empty();  trail.full();  trail.capacity();   // capacity() is static: 4
trail.front();  trail.back();  trail.pop_back();  trail.clear();
for (const Vec2& p : trail) { /* only the items in use */ }
List<int, 3> three{1, 2, 3};   // initializer list; more than N items throws like push()
CallWhen it throwsException and message
push(v), initializer listthe list is fullstd::length_error: aether::List<T, 2>: the list is full (2 items) and never grows. Erase an item first, or use a larger N
operator[], erase(i)i >= size()std::out_of_range: aether::List<T, 2>::operator[]: index 1 but the list has 1 items
front(), back(), pop_back()the list is emptystd::out_of_range (same form, naming the call)

Slots past size() are always kept zero, so an item you erased can never show up as a change. In the row, a list is the field trail.count (a uint8 when N is at most 255, otherwise a uint16) followed by trail[0] to trail[N-1]. Every slot is part of the row whether it is used or not, so a List<T, 16> of 20-byte items is 321 bytes in every row. When such a row is sent as a message only the count and the used items travel, but in a view row all slots count.

When a list is the wrong tool. A collection with no natural limit (an inventory, a mailbox, a friend list) is not a list: make one entity per item, with an owner member. The recipe is below.

Parts and arrays

A nested struct is a part: its members sit in the row in place. A std::array is N copies of its element. A list of parts is a count and N copies of the part. You use them like any C++ member: sheep.position.x, sheep.colour[1], sheep.trail[0].y.

The flattened field names are the names the rest of Aether uses: in select("position.x") of a view, in spatial("position"), in delta("mood", Tier::Low), and in the layout sent to clients. For Sheep they are, in order:

name  position.x  position.y  colour[0]  colour[1]  colour[2]
trail.count  trail[0].x  trail[0].y  trail[1].x  trail[1].y  trail[2].x  trail[2].y  trail[3].x  trail[3].y
mood  shorn

That is 17 fields and 102 bytes in a packed row (the struct itself is 120 bytes in memory because of alignment padding).

Limits

LimitValueWhat happens beyond it
Members per struct64 by reflection (kMaxMembers)Compile error: aether::reflect: this struct has more than aether::reflect::kMaxMembers (64) members. Split it into nested structs. The optional macros take at most 32 members per struct.
Flattened fields per row1024 (every array item and list slot counts)With the optional macro: compile error the row has more than 1024 flattened fields. For a struct found by reflection this version does not check it when the component is registered; keep to 1024, because the .aether tool treats a message above it as unusable as a view row (canBeViewRow false).
Membersat least oneAn empty struct is not a component (a compile error that suggests a local component). For a component with no data use a tag: hand-built types.
Alignmentat most 64Compile error: a component struct is aligned to at most 64 bytes
Copyingtrivially copyableCompile error: a component struct is stored as it is and copied byte for byte: it must be trivially copyable
A held input or an action255 bytes packed, every List counted fullCompile error naming FitsIn255<..., YourStruct, 641> with the packed size
A message a client sends4096 bytes by default (Options::maxMessageBytes)The server closes the connection of a client that sends more
A message the server sends16 KiB (one eighth of the 128 KiB output buffer); no schema message may exceed 16384 bytesthe .aether tool refuses the file

What is refused, and what you see

A struct that cannot be a row is refused at compile time, with a message that says what to do. The wording below was produced by compiling each case with GCC 15.

You wroteWhy notWhat you see (first error)
std::string, std::vector, std::map, std::optional, std::string_view, a pointer, a smart pointer, char, long double, an unscoped enum with no fixed typeno fixed size, or owns memory elsewhereFor registry.component<S>(): ecs::Registry::component<S>(): S is not a flat struct, so it cannot be a component. A component is a plain struct with at least one member, every member a fixed size: ... For anything else (std::string, std::vector, a pointer, a class with a constructor or private members) use a local component: registry.local<S>(). For view<S>(): ecs::View::define<S>(): S is not a flat struct, so it cannot be a view row: ... To show part of a local component, build the row with view.from(local, function).
the same member in a message, input or actionaether::reflect: a member of this struct has a type that a row or a message cannot hold. The struct, the member name and its type are in the first 'In instantiation of ... MemberCheck<Struct, FixedString<N>{"member"}, Type, false>' line ... Allowed: int8_t..int64_t, uint8_t..uint64_t, an enum with a fixed underlying type (enum class State : std::uint8_t), float, double, bool, aether::String<N>, ecs::Entity, a plain struct, std::array<T, N> and aether::List<T, N> of those. Not allowed: char, std::string, std::vector, pointers, C arrays (use std::array<T, N>). A build with -std=c++26 names the struct, member and type in the error line itself.
a C array member, a reference memberuse std::arrayaether::reflect: this struct has a member that is a C array (use std::array<T, N>), a reference, or a member that cannot be default constructed (a class without a default constructor)
a bit-fieldcannot take its addressaether::reflect: this struct has a bit-field member (int flags : 3). A bit-field cannot be reflected: use a whole integer member (std::uint8_t, ...) or a bool instead
a constructor, a private member, a virtual functionnot an aggregateaether::reflect: this struct is not a plain struct (an aggregate): it has a constructor, a private or protected member, a virtual function or a base class that is not public. ... (or ecs: this type is not a plain struct, so it cannot be a row. Make it a struct with public members and no constructors, depending on the call)
a base class, or a member that can be built from anything (std::any)reflection cannot tell them apartaether::reflect: this struct has a base class, or a member whose type can be made from anything ... put the base in a member, or flatten it into this struct
a List of Listone level of count onlyaether: a List of Lists cannot be a row. Put the inner List in a struct and make a List of that struct, or use std::array<List<T, N>, M>
a struct declared inside a functionmember names cannot be read from a local typeaether::reflect: this struct is declared inside a function, so its member names cannot be read. Declare the struct at namespace scope (outside every function, in your game's namespace or in a header)
more than 64 memberssee Limits
Trap: a struct that is not flat is silently local. spawn(Sheep{...}) turns a flat struct into a flat component, but turns any other type into a local component (kind 3), with no error. If you add a std::string to a row you meant to send, spawn keeps compiling; the mistake only shows when something asks for the row to be flat, such as view<Row>(), which is a compile error. Declare every row that must be flat with view<Row>() (or registry.component<Row>()) so the compiler checks it.

How a flat row is stored

A flat component is stored as the struct: sizeof(S) bytes per row, aligned as the struct is aligned, padding included. That is why get and queries hand out S&. The packed form (little-endian, members in declaration order, flattened leaves, no padding) is what travels: the wire, view rows, saved records and the type's fingerprint. The two are different, so never assume the memory layout of a struct equals its wire layout. A row of a flat component can also be read as another struct with the same members in the same order; that works through a slower copy.

Writing to a row never costs anything and never has to be announced: the registry finds changes by comparing the new bytes with the previous commit, so a row that you only read is not sent and not saved, and a write that leaves the value as it was is not a change.

Reading and writing a row by path

With Typed<S> you hold S& and use the members. When you hold a Component (the untyped handle) you get a Row, and you reach a member by a path of member pointers, with an integer going into a std::array or List:

ecs::Component c = sheep;                       // a Typed<Sheep> converts
ecs::Row row = registry.get(e, c);
row.set(&Sheep::mood, Mood::Angry);
row.set(&Sheep::position, &Vec2::y, 8.0);       // a path: member of a member
row.set(&Sheep::colour, 1, std::uint8_t{200});  // an index into the array
const double y = row.get(&Sheep::position, &Vec2::y);
const Sheep whole = row.get<Sheep>();            // the whole struct at once
row.set(whole);

An index past the end of the array throws std::out_of_range (ecs: array index out of range in a row path; for a list, ecs: list index out of range in a row path). A path whose first member is not a member of the row's struct is a mistake that a debug build stops on with a message naming both structs; a release build does not check.

Next, a way to define a component when there is no struct to write.

Kind 2: the hand-built Type component

A struct is the right tool when you know the fields while you write the code. Sometimes you do not: a modding layer defines new kinds of object from a data file, a tool builds components from a script, or you simply need a marker with no data. For those, you describe the fields at run time with ecs::Type::define.

examples/meadow/shepherd_type.cpp
#include <aether/ecs.h>
#include <print>

using namespace aether;

int main() {
    ecs::Registry registry;

    const ecs::Type shepherd = ecs::Type::define("Shepherd")
        .uint64("id").key()               // a key: needed to save the component
        .string("name", 16)
        .uint16("flockSize")
        .build();

    ecs::Component component = registry.component(shepherd);

    const auto name  = shepherd.field<std::string_view>("name");     // a handle, looked up once
    const auto flock = shepherd.field<std::uint16_t>("flockSize");

    ecs::Entity e = registry.create();
    ecs::Row row = registry.add(e, component);                       // a zeroed row
    row.set(name, "Ayla").set(flock, 12);                            // set() chains
    std::println("{} keeps {} sheep", row.get(name), row.get(flock));
    std::println("{} bytes, fingerprint {:x}", shepherd.blockLength(), shepherd.fingerprint());
}

Notice three things. A row is a Row: a pointer to bytes. You do not read the bytes yourself, you read a field through a Field<T> handle that you looked up once by name. And there is no Shepherd struct in the program at all.

Field kinds

The builder has one method per field kind. Each adds a field after the previous one. The kind decides the width and the C++ type you read and write it as.

Builder methodPrim (value)WidthField<T>: T
int8(name)Int8 (11)1std::int8_t
uint8(name)UInt8 (12)1std::uint8_t
int16(name)Int16 (13)2std::int16_t
uint16(name)UInt16 (2)2std::uint16_t
int32(name)Int32 (0)4std::int32_t
uint32(name)UInt32 (3)4std::uint32_t
int64(name)Int64 (1)8std::int64_t
uint64(name)UInt64 (4)8std::uint64_t
float32(name)Float32 (14)4float
float64(name)Float64 (5)8double
boolean(name)Bool (6)1bool
enumeration(name, allowed = {})Enum (7)4 (an int32)std::int32_t; the list of allowed values is part of the type and its fingerprint
string(name, bytes)FixedStr (10)bytes (at least 1), zero paddedstd::string_view

These are the same kinds a flat struct produces: a struct's double becomes Float64, its String<16> becomes FixedStr of 16 bytes. Prim values 8 and 9 are the variable-length kinds that exist in the database layer and that a row never has: a row has one size.

Key, tag and since

Three modifiers apply to the field you added last:

  • key() marks it as the key (at most one). It must be an integer or a string of at most 16 bytes. A component can be saved only if it has a key. A floating-point field cannot be the key.
  • tag(n) sets the field's durable tag, its identity across versions of the type. The default is its position in the declaration.
  • since(version) records the version of the type that added the field (default 1). Type::define(name, version) sets the type's own version (default 1).

build() checks the definition and throws std::invalid_argument naming the type and the field:

MistakeMessage
two fields with one nameecs::Type 't': field 'a': duplicate name
two fields with one tagecs::Type 't': field 'b': duplicate tag 3
a second keyecs::Type 't': field 'b': a second key (the first is 'a')
a key that is a float, or a string longer than 16ecs::Type 't': field 'a': a key must be an integer or a string of at most 16 bytes
key(), tag() or since() before any fieldecs::Type 't': key() before any field
string(name, 0)ecs::Type 't': string field 'a' needs at least 1 byte

Looking a handle up can fail too. field<T>(name) throws std::invalid_argument if there is no such field (ecs::Type 'Shepherd': there is no field 'nope') or if T is not the C++ type of its kind (ecs::Type 'Shepherd': field 'name' is string(16), not double: use field<string_view>("name")). For several at once: auto [x, y] = position.fields<double, double>("x", "y");. Asking for a Field<T> with a T that is the C++ type of no kind is a compile error: Field<T>: T is not the C++ type of any field type.

Rows are packed

The fields of a hand-built type are laid out packed: little-endian, in declaration order, with no padding. The Shepherd row above is 8 + 16 + 2 = 26 bytes. Because nothing is aligned, fields are read with unaligned loads, which is why you go through a Field and never cast the row pointer yourself. A hand-built component is not stored as a C++ struct (type.store() is null), so registry.get<S> and S& in queries are not available for it: you work with Row and ConstRow.

A flat struct produces exactly the type you would write by hand. struct Position { double x, y; } has the same fields, offsets, widths and fingerprint as Type::define("Position").float64("x").float64("y").build(); the struct is only stored differently in memory.

fingerprint() is a 64-bit FNV-1a hash of the type's name and, for each field in tag order, its tag, kind, since-version, key flag, name, enum values and string width. Two types with the same fingerprint are the same layout. It is what the database stores next to a topic to notice that a saved struct changed (see Saving).

Tags

A type with no fields is a tag: a component that says "this entity is in some state" and carries no data. A tag stores only the list of entities that have it.

auto scared = registry.component(ecs::Type::define("scared").build());
registry.add(sheep, scared);                                   // the Row it returns must not be read or written
registry.query(position, ecs::none(scared)).each([](ecs::Entity e, Vec2& p) { /* a sheep that is not scared */ });

A tag has no key, so it cannot be saved, and it cannot be a source of a view (build() throws 'scared' is a tag: it cannot be a source). Use it in queries as a required component, with ecs::none(...) or with ecs::any(...); The registry page covers queries.

Trap: a comment disagrees with the code. The comment on TypeBuilder::build in Type.h lists "no fields" among the things that throw. The code does not throw: a type with no fields is a tag, as the same header says in its introduction.

Registering and naming

registry.component(type, reserveRows = 1024) registers it. A component's name must be unique among components and views in one registry; a second registration of the name throws ecs::Registry: a component or view named 'Shepherd' exists. The second argument is the number of rows to allocate up front. registry.component(type, ecs::persist("topic")) makes it a saved component (Saving); it throws if the type has no key field.

If a hand-built component and a struct have the same name, Aether refuses to treat one as the other: registry.component<S>() on a name already taken by a hand-built type throws ecs: two different structs are both called 'Vec2': a component built by hand from a Type and Vec2 ...; and using the struct where its rows are packed throws ecs: the component 'X' was made by hand from a Type, so its rows are not stored as the struct ... and cannot be used as one.

Kind 3: the local component

So far every component had to be a block of fixed-size fields, because clients and the database must be able to read it. But a server keeps things that are nothing like that. A path-finding graph. A std::map from sheep to the time it was last seen. A std::shared_ptr to a navigation mesh that many entities share. A class with a constructor that opens something.

Those are local components: any movable C++ type, kept on the server, never sent and never saved by the registry. They are full citizens of the registry otherwise: they attach to entities, show up in queries, fire hooks and can be added later in a deferred way.

Suppose the meadow has a sheepdog, and the world keeps a Herd that tracks every sheep it has seen and which are in the pen:

examples/meadow/Herd.h
#pragma once
#include <cstdint>
#include <map>
#include <vector>

#include <aether/shared.h>

namespace meadow {

struct Herd {                                          // server only: not a plain struct of fixed-size members
    std::map<aether::ecs::Entity, double> lastSeen;    // sheep -> game time it was last in sight
    std::vector<aether::ecs::Entity> pen;              // the sheep that are penned
};

struct HerdInfo { std::uint32_t sheep, penned; };      // what clients may see of it: a plain, flat struct

}  // namespace meadow

You attach it like any component. Inside a server::Game:

examples/meadow/World.h (excerpt)
void onStart() override {
    auto herd = local<meadow::Herd>();                         // a Local<Herd> handle
    zone_ = spawn(meadow::Herd{});                                // or: registry().add(zone, herd, meadow::Herd{})
}

void update(float) override {
    meadow::Herd& herd = get<meadow::Herd>(zone_);              // the object itself
    herd.lastSeen[someSheep] = time();
}

Notice that spawn needed no registration: a type that is not a flat struct becomes a local component automatically, and get<Herd> gives it back as a Herd&. At registry level it is registry.local<Herd>() and registry.get(zone, herd).

The rules

  1. The type must be movable, and its move constructor and destructor must be noexcept. The registry keeps objects in one contiguous table and moves them when the table grows or an entity is removed. If a move could throw, the table could be left half moved. The standard containers (std::map, vector, string, shared_ptr, unique_ptr, unordered_map) already qualify. Your own class qualifies if you do not declare a move constructor, or declare it noexcept.
    struct Cache {
        Cache(Cache&& other) { items = std::move(other.items); }          // not noexcept: refused
        std::vector<int> items;
    };
    // fix: Cache(Cache&& other) noexcept { ... }
    //  or keep the throwing part behind a std::unique_ptr member
    The compiler stops with: local<T>(): T's move constructor must be noexcept, because the registry moves T when its storage grows. Add noexcept to T(T&&), or hold the throwing part in a std::unique_ptr. A type that cannot be moved at all gets: registry.local<T>(): T must be movable (add(e, handle, T{...}) moves it in). Hold a type that cannot move through a std::unique_ptr member.
  2. Alignment is at most 64 (the registry's buffers are 64-byte aligned), and the type is a plain class type (not a reference, a const type or a C array).
  3. It is never sent and never saved by the registry. registry.local<T>(ecs::persist(...)) is a compile error: a local component stays on the server and is never persisted; persist a flat plain struct (registry.component<S>(persist(topic, &S::key))) with the part of it that must last. It does not survive a restart or a hand-over to another process. A server-only row can still be saved through the game's own storage; see Saving.
  4. The object lives in the table. One row per entity, of sizeof(T) rounded up to alignof(T), inside the component's 64-byte aligned buffer. It is a live object: growth, removal and group changes move it with its move constructor and never copy its bytes. It is destroyed when the component is removed, the entity is destroyed, or the registry is destroyed.
  5. A reference is valid only until the next structural change. A T& (like a Row or an S&) is invalidated by an add, remove or destroy of this component on any entity, because the object may be moved to a new row. Do not keep it across ticks.
    meadow::Herd& herd = get<meadow::Herd>(zone_);
    spawn(meadow::Herd{});             // another entity gets a Herd: the table may move
    herd.lastSeen.clear();             // WRONG: herd may now point at moved-from memory
  6. Default construction is optional. registry.add(e, handle) builds a default object and needs a default constructor (registry.add(e, handle): T has no default constructor; give the value: registry.add(e, handle, T{...})); registry.add(e, handle, T{...}) moves a value in.
  7. The name is the compiler's name for the type (Herd, or game::Herd for a type in a namespace). Calling local<T>() twice gives the same handle. A name already used by a component or view throws ecs::Registry::local<Herd2>: a component or view named 'Herd2' exists.
  8. Missing is an error you can see. get on an entity that has none throws std::logic_error (ecs::Registry::get: entity 0:1 has no 'meadow::Herd': add it first, or use has(e, handle) / tryGet(e, handle)); tryGet returns a pointer or nullptr.

A local component is a normal part of queries: registry.query(position, herd).each([](ecs::Entity e, Position& p, Herd& h) { ... }) gives Herd&, ecs::read(herd) gives const Herd& and ecs::optional(herd) gives Herd*. Its hooks get the object: registry.onAdd(herd, [](ecs::Entity, Herd&) { ... }). The ECS page covers queries and hooks.

Why a local component cannot be a view row

You might expect to write view.select(herd, "lastSeen") and have clients see the map. That is a compile error, on purpose, and the reason is how a view row is built:

  • A view row is a fixed-size block of fields, each with a kind and a width. The view copies a field from its source with a plain memcpy of width bytes from a known offset in the source row to a known offset in the view row.
  • To send only what changed, the registry compares the bytes of the row with the same entity's row at the previous commit, and records the difference as a change.
  • It keeps the last few commits of rows in a history ring, one fixed-size copy per commit, so a client that missed some can be sent the difference from the row it has.
  • It sends clients a layout, one entry per field: name, kind, width, flags. The client decodes every row by that layout, with no knowledge of your C++ types.
  • A saved row has a fingerprint of its fixed schema, so a change to the struct is noticed.

A std::map or a std::shared_ptr has no field kind, no width and no stable byte form. Its bytes are pointers into the heap, so copying them is meaningless, comparing them says nothing about the content, and a client could not decode them. A local component's object is not trivially copyable either: the pool moves it with its constructors, never with a byte copy.

So the SDK gives you one explicit bridge instead of a half-working shortcut: you write a function that fills a plain row from the object. You decide what clients see, and the result is an ordinary flat row, so delta, history, visibility and the client work unchanged.

The bridge: function views

A function view is a view whose rows are filled by a function. Declare the row as a plain struct (HerdInfo in Herd.h above), then give the view the source and the function:

examples/meadow/World.h (excerpt)
void onStart() override {
    auto herd = local<meadow::Herd>();
    view<meadow::HerdInfo>().from(herd, [](const meadow::Herd& in, meadow::HerdInfo& out) {
        out.sheep  = static_cast<std::uint32_t>(in.lastSeen.size());
        out.penned = static_cast<std::uint32_t>(in.pen.size());
    }).global();                                              // visibility: everyone sees it
}

The rules, all of them:

RuleDetail
One row per source entityThe function runs for every entity that has the source component. There is no way to give one entity many rows: make one entity per item instead.
When it runsAt every commit (once per tick), for every source entity, before the registry compares rows. Keep it short and do not allocate.
SourceA local component (the function takes const T&) or a flat struct component (the handle of registry.component<C>(): the function takes const C&, a const reference to the stored row).
Signaturevoid(const Source& in, Out& out) or bool(const Source& in, Out& out). Out is a plain flat struct taken by non-const lvalue reference; in is an lvalue reference of exactly the source type. Use a lambda with explicit parameter types, a functor, or a plain function. A generic lambda (auto parameters) or an overloaded function name cannot be inspected and is a compile error.
out starts zeroedEvery commit the function gets a zeroed Out. Anything you do not set is zero.
Returning boolfalse means "this entity has no row in the view this commit". Use it to hide an entity.
ThrowingA function that throws drops that entity's row for that commit. It is reported once, until it stops throwing, through registry.onError(fn) or, if none is set, one line on stderr: ecs::View 'HerdInfo': the function threw for entity 0:1 (boom); its row is left out of the view until it stops throwing. The tick goes on.
AloneA function view has one source and no select, join or groupBy next to it; build throws a function view has one source and no select, join or groupBy next to it in this version.
Everything else is a normal viewEvery member of Out is a delta field, the view is subscribable, and visibility (spatial, global, ownerOnly, show), every, history, priority work as for any view. See Views and visibility.
Rows are not savedViews are never saved; only components are.
Trap. The comment on from(Typed<C>, fn) in View.h says the function gets "a copy of the row". The code hands it a const reference to the stored row (*reinterpret_cast<const C*>(src)). Do not rely on it being a copy, and never keep the reference.

The low-level form on a registry is ecs::View::define<Out>().from(local, fn).build(registry); ViewBuilder::fromFunction(component, rowType, rowName, fn) is the byte-level form the two templates are built on. Both are in the Reference.

The sheep and the herd show the whole idea: the Herd is whatever the server needs it to be, and HerdInfo is the little flat summary that goes to clients.

Reading and writing rows on the server

In a server::Game you rarely touch the registry. You say what an entity has with spawn, and you read and change it with get and set. These work the same for every kind of component.

examples/meadow/World.h (excerpt)
void onJoin(Player& p) override {
    p.entity = spawn(Walker{0, 0, 60},                          // a flat row: a component
                     meadow::Herd{});                           // a non-flat type: a local component
}

void update(float) override {
    for (Player& p : players()) {
        Walker& w = get<Walker>(p.entity);                    // writable reference to the row
        w.x += 1;                                               // that is all: nothing to write back
        const Walker& peek = get<Walker>(p.entity, READONLY);  // only looking
    }
}

spawn

spawn(rows...) creates an entity and gives it one row per argument. A flat struct becomes a flat component (it is registered the first time, if view<Row>() has not done so); any other type becomes a local component. If a row is declared saved with persist<Row>(), it is added with its key; a key that is empty, zero, too long or used by another live row throws (std::invalid_argument or std::logic_error, naming the row) and no entity is left behind. The entities a client's onJoin spawned are removed if onJoin throws. It returns the new ecs::Entity.

get and READONLY

get<Row>(e) returns Row&, a reference to the row itself, for a view row, a local component or a saved row alike. Reading costs nothing: a row you only looked at is not sent and not saved, because changes are found by comparing. get<Row>(e, READONLY) returns const Row&: the compiler then refuses a write, and the call says to the next reader that you only look. READONLY needs no using and no namespace inside a Game.

The reference is good until the entity gains or loses that component, or is destroyed. Do not keep it past the tick, and do not keep it across a spawn or destroy that may move the table (the same rule as local components).

get throws std::logic_error, in every build, when it cannot give you the row:

server::Game::get<Purse>: entity 0:1 has no Purse row (spawn it with the row, or check what the entity was given)
server::Game::get<Walker>: entity 0:1 is dead or was never created (it may have been destroyed; do not keep its id)

set

set(e, row) assigns a whole row. It throws the same errors as get. For a local component the object is assigned, so its type must be copy assignable (server::Game::set(e, value): the local component is assigned, so its type must be copy assignable; change it in place with get<T>(e) instead, a compile error otherwise).

A change that is a jump (a portal, a respawn) can be marked so that clients treat it as a discontinuity instead of smoothing across it: snap(e, row) is set followed by snap(e). See Client.

Recipe: an inventory is entities

A list is bounded on purpose. A collection with no limit (an inventory, a mailbox, a friend list) is made of entities, one per item, each with an owner. Adding an item is a spawn, removing one is a destroy, and there is no size to outgrow.

examples/meadow/Shared.h (addition)
struct Item { ecs::Entity owner; std::uint32_t kind; std::uint16_t count; };
inline constexpr std::uint32_t kEgg = 1;      // item kinds are yours to choose
examples/meadow/World.h (excerpt)
void onStart() override {
    view<Item>();                                   // no spatial, global or ownerOnly: sent only to whoever is shown it
}

void onJoin(Player& p) override {
    p.entity = spawn(Walker{0, 0, 60});
    const ecs::Entity egg = spawn(Item{p.entity, kEgg, 3});
    p.show(egg);                                    // this player sees it, wherever it is; nobody else does
}

void give(Player& p, std::uint32_t kind) {
    p.show(spawn(Item{p.entity, kind, 1}));
}

void take(ecs::Entity item) { registry().destroy(item); }   // gone for every client that had it

On the client, an inventory is the items whose owner is you: session.view<Item>().each([&](ecs::Entity e, const Item& it) { if (it.owner == session.me()) draw(it); }). How show and the default visibility work is on Views and visibility.

Next: the optional macros, then the names that components, views and messages get.

The optional macros

A plain struct needs none of this. The macros are an optional fallback, for four situations:

  • A name that differs from the struct's: AETHER_VIEW(S, "crates", members...) gives the view a name of its own, written next to the struct.
  • A key inside the struct: key(member) in the list marks the persisted key, instead of passing it to persist.
  • A fixed id for a message or an action that something other than the id table must know: AETHER_MESSAGE(S, 30, ...), AETHER_ACTION(S, 5, ...).
  • Per-member handles: a struct with a macro also gets one Field per member, so registry.add(e, crate).set(crate.position.x, 3.0) works.
struct Vec2  { double x, y; };
AETHER_STRUCT(Vec2, x, y)                          // a part: can be a member of a row

struct Crate { Vec2 position; std::uint32_t items; String<8> label; };
AETHER_VIEW(Crate, "crates", position, items, label)     // the view is called "crates"

struct Door  { std::uint64_t id; std::uint8_t open; };
AETHER_COMPONENT(Door, key(id), open)               // id is the persisted key

struct Wave  { std::uint8_t strength; };
AETHER_ACTION(Wave, 5, strength)                    // an action with id 5

struct Ping  {};
AETHER_MESSAGE(Ping, 30)                            // a message with id 30, no members
MacroMeaning
AETHER_COMPONENT(S, m...)Describes S by the listed members, in the listed order. key(m) in the list marks the key member.
AETHER_STRUCT(S, m...)The same, for a nested part. (It is defined as AETHER_COMPONENT.)
AETHER_VIEW(S, "name", m...)AETHER_COMPONENT plus the view's name. view<S>() uses that name.
AETHER_INPUT(S, m...)AETHER_COMPONENT plus: this is the held input (kind 0).
AETHER_ACTION(S, id, m...)The action id must be 1 to 255 (the id is 1..255 (0 is the held input)). No members: an action with no payload.
AETHER_MESSAGE(S, type, m...)The type must be 16 to 255 (the type is aether::kFirstGameType (16) .. 255).
AETHER_COMPONENT_EMPTY(S)For a struct with no members.

Rules that the macros enforce at compile time:

  • Put the macro at namespace scope, after the struct and in the same namespace (it defines functions that are found by argument-dependent lookup).
  • List every member of the struct, once each, in the order of the row: AETHER_COMPONENT(S): the list and the members of S differ. List every member of S, once each, in the order of the row.
  • At most 32 members in the list; at most 1024 flattened fields (the row has more than 1024 flattened fields (every item of a std::array and every slot of a List counts as fields). Make the arrays and lists smaller, or keep part of it out of the row).
  • A member of a type a row cannot hold: AETHER_COMPONENT(S): member 'm' has a type that a row cannot hold. Allowed: ... Not allowed: std::vector, std::map, pointers, C arrays (use std::array). And a List of Lists: AETHER_COMPONENT(S): member 'm' is a List of Lists, which a row cannot hold. Put the inner List in a struct ... or use std::array<List<T, N>, M>.

A struct that has a macro is described by the macro, never by reflection: the members listed, in the order listed. Every other struct is described by reflection. Do not add a macro "just in case": if the list and the struct drift apart, the compiler stops you, but the plain struct cannot drift at all.

Names: namespaces and collisions

Aether names things by the struct's name, without its namespace. That keeps the wire small and the client simple, and it means that names live in one flat space per registry:

ThingIts name
A component from struct meadow::SheepSheep
A view of itSheep, or the name you pass: view<Sheep>("flock"), or the name in AETHER_VIEW
A local componentthe compiler's name for the type, so Herd or game::Herd for a type in a namespace
A message or actionthe struct's name; the server announces its table of names and ids at connect

So put your game in a namespace, as meadow does: it costs nothing on the wire. And remember that two different structs with the same unqualified name collide. The registry refuses the second one when it is registered:

ecs: two different structs are both called 'Hero': a::Hero and b::Hero. A component is named after its struct without the namespace, so they would share one row. Rename one of the structs, or give the view of one a name of its own: view<b::Hero>("Hero2") (that name is then the component's too)

More name rules:

  • A component name and a view name must be unique in one registry. A view and the component of its own struct may share a name (that is the default); anything else throws ecs::View 'Hero': a component or view of that name exists or ecs::Registry: a component or view named 'Flat' exists.
  • A struct must be declared at namespace scope. A struct inside a function cannot be reflected (the error is above), and a test body is a function: declare test structs outside the test.
  • Do not name a struct like an SDK type. With using namespace aether; a struct called List, String, Contract, Rows or Flags (or Entity, with using namespace aether::ecs;) makes the bare name ambiguous (reference to 'List' is ambiguous). The schema tool warns about it.
  • Give a player's row a name that is not an SDK type: Walker, Hero, Car.

The contract file

The structs you have seen so far have to be written twice if the server and a client are separate programs, and kept in agreement by hand: a shared header does that for C++, nothing does it for a web page. The contract file removes the problem. You write the data once in a small language, in a file with the extension .aether. The tool aether-schema generates from it a C++ header, a JavaScript module, a TypeScript declaration and a JSON description, and a fingerprint of the whole file that a server and a client compare when they connect.

It is optional. The generated structs are ordinary C++ structs; you may write them by hand in a shared header instead. What you give up is the single source for non-C++ clients, and the check that both sides were built from the same version.

The language

examples/pasture/pasture.aether
// pasture.aether: a larger contract
namespace pasture;

enum Mood : u8 { Calm, Grazing = 5, Angry }

message Vec2 { f64 x; f64 y; }

message Sheep {
    string<16> name;
    Vec2 position;
    array<u16, 4> hotbar;
    entity herder;
    Mood mood = Angry;
    f32 wool = 1.5;
    u32 age = 3;
    bool alive = true;
}

message Flock { list<Vec2, 8> members; }
message Shear = 20 { entity target; }
message Whistle {}

The grammar, in full:

file     = "namespace" name {"::" name} ";" { enum | message }
enum     = "enum" Name ":" int-type "{" Member ["=" int] {"," Member ["=" int]} [","] "}"
message  = "message" Name ["=" id] "{" { type name ["=" default] ";" } "}"
type     = u8 u16 u32 u64 i8 i16 i32 i64 f32 f64 bool entity
         | string<N> | list<type, N> | array<type, N> | EnumName | MessageName
default  = number | true | false | EnumMember
comments = // to the end of the line

The rules:

  • The file starts with its namespace. namespace pond; or namespace a::b;, exactly once. Without it: a schema starts with its namespace. Write 'namespace mygame;' before the first declaration. Enums and messages may come in any order, and a message may use a type declared later; the C++ is written in dependency order. A ; after a closing brace is accepted.
  • Comments are // only. A block comment is an error: block comments are not part of the language. Write each comment line with '//'.
  • One field per declaration. f32 dx, dy; is an error: one field per declaration: write 'f32 dx;' and then the next field on its own line.
  • A message may be empty (message Whistle {}): a one-shot signal. It cannot be a view row (it has no fields).
  • At most 64 fields per message (the message A has 65 fields; the SDK's reflection takes at most 64 members. Group some into a nested message), and a message may be a field of another. A cycle is an error: the message A contains itself (A -> B -> A), so it would have no size. Remove one of the members.
  • Names are unique across messages and enums: 'A' is declared twice (first at line 1). Rename one of them. A field may not be named like a message or enum of the file, because C++ forbids it inside the struct: the field 'B' has the name of a message of this schema ... Start field names with a lowercase letter, for example 'b'. An unknown type is an error (the tool suggests the nearest).

Types

Contract typeC++JavaScriptWire
u8 u16 u32 u64std::uint8_t ... std::uint64_tnumber; u64 is BigInt1, 2, 4, 8 bytes, little-endian
i8 i16 i32 i64std::int8_t ... std::int64_tnumber; i64 is BigInt1, 2, 4, 8 bytes
f32, f64float, doublenumber4, 8 bytes
boolboolboolean1 byte
entity::aether::ecs::EntityBigInt8 bytes
string<N>::aether::ecs::String<N>string; N UTF-8 bytes, zero padded, never truncated (a RangeError if too long or if it contains a NUL)N bytes
list<T, N>::aether::ecs::List<T, N>arraya count (1 byte if N is at most 255, else 2) and the items in use
array<T, N>std::array<T, N>array of exactly N itemsN items
an enumenum class Name : std::uint8_t (its integer type)a frozen object of members, and the value is a numberthe integer
a messagea nested structa nested objectits fields in place

N in string, list and array is from 1 to 65535 (string<0>: N is the size in bytes and must be from 1 to 65535. Write for example string<16>). A list of lists is rejected: a list of lists cannot be sent by the SDK. Put the inner list in a message and make a list of that message, or use array<list<T, N>, M>.

Enums

An enum needs an integer type: enum Mood : u8 { Calm, Grazing = 5, Angry }. Members number themselves from the previous value plus one, starting at 0, unless you write = value (decimal, negative, or hex like 0x1F). So Angry above is 6. A trailing comma is allowed. Every value must fit the type (the value of E::A does not fit in u8 (0 to 255). Use a wider type, e.g. 'enum E : u16'), no two members may share a value (the enum members 'A' and 'B' of E have the same value 1. Give each member its own value), and there must be at least one member. An enum becomes an enum class with that integer type.

Defaults

A number, bool or enum field may have a default: f32 wool = 1.5;, bool alive = true;, Mood mood = Angry;. An integer field takes a whole number that fits (the default of 'lo' (200) does not fit in i8, which holds -128 to 127); a float field takes an int or a float; a bool takes true or false; an enum takes a member name. A string, list, array, entity or nested message cannot have one: only a number, bool or enum member can have a default value; the field 'x' is string. Remove '= 1'. A default becomes a member initializer in C++ (the struct is still an aggregate) and makes the field optional in the JavaScript encode. A default does not change the fingerprint.

Fixed ids

A message may carry an id: message Shear = 20 { ... }. Ids are 16 to 255 and unique (the wire id 3 of message A is out of range. A game's message ids are 16 to 255 (0..15 belong to aether). Write 'message A = 20 { ... }', or leave the id out to let the server assign one by name; the wire id 20 is used by both A and B. Give each message its own id). Without an id, the server assigns one in registration order and announces the table of names to each client, so nothing needs to be hashed or can collide. Use a fixed id only if something that cannot read that table (a tool, another language) must know it. A message with a fixed id gets the line AETHER_MESSAGE(Shear, 20, target) in the C++ and id === 20 in the JavaScript, and, because it is written with the macro, may have at most 32 fields. The tool does not check this: a fixed-id message of 33 or more fields generates a header that does not compile ('AETHER_COMPONENT_DECL' has not been declared). Give such a message no fixed id, or split it into parts.

Reserved names

RefusedBecause / example error
C++ keywords and common macros: NULL EOF assert errno stdin stdout stderr INT_MAX INT_MIN unix linux major minorthey would break the generated header
JavaScript reserved words and the globals the module uses as type names: Object Array Number BigInt String DataView Map Set Date Boolean Function Promise Reflect Proxy ...the message name 'Object' is a JavaScript global the generated module uses. Use another name, for example 'Object_x'
a name with __, a leading underscore, or the prefix aether_the field name '_x' is reserved (a leading underscore is reserved for the implementation in C++). Use another name, for example '_x_x'
std, aether, string, size_t as type namesreserved

A message or enum called List, String, Contract, Rows, Flags or Entity is a warning, not an error: the message List has the name of aether::List: where a file says `using namespace aether;` ... a bare `List` is ambiguous ....

Size checks

The tool computes the biggest a message can be, with every list and string full, and checks it. Above 16384 bytes it is an error (the message A is up to 24002 bytes packed (every list full) but a message may be at most 16384 bytes. Give its list<T, N> or string<N> a smaller N). Above 4096 it is a warning first (the server's default maxMessageBytes is 4096, so a client cannot send it unless the server raises that option), and an error when it generates, unless you raise the limit in two places: pass --max-message-bytes n (or MAX_MESSAGE_BYTES n to aether_schema()) to the tool and set the same Options::maxMessageBytes on the server. A message that is used as a held input or an action must also fit 255 bytes (fitsInputOrAct in the JSON).

Every error has the form file:line:col: error: what is wrong. what to write. The first syntax error stops the parse; semantic errors are collected and all printed. A schema with an error fails the build with those lines.

What it generates

For the file above, the C++ header pasture/Messages.h contains (excerpt):

pasture/Messages.h (generated)
namespace pasture {

enum class Mood : std::uint8_t {
    Calm = 0,
    Grazing = 5,
    Angry = 6,
};

struct Sheep {
    ::aether::ecs::String<16> name;
    Vec2 position;
    std::array<std::uint16_t, 4> hotbar;
    ::aether::ecs::Entity herder;
    Mood mood = Mood::Angry;
    float wool = 1.5f;
    std::uint32_t age = 3U;
    bool alive = true;
};

struct Shear {
    ::aether::ecs::Entity target;
};
AETHER_MESSAGE(Shear, 20, target)

inline constexpr ::aether::Contract contract{"pasture.aether", 0x8d4ba7b34285e268ULL};

}  // namespace pasture

The structs are in dependency order, and the header includes only <aether/shared.h>. A game includes it from its shared header and brings in the names it uses: #include <meadow/Messages.h> then using meadow::Walker;.

With --js, --dts and --json the tool also writes:

  • pasture.js, a plain ES module, little-endian. It exports namespace, contractFingerprint (a BigInt), each enum as a frozen object, and one codec per message: { name, id, fingerprint, wireFingerprint, minByteLength, maxByteLength, byteLength(v), encode(v) -> Uint8Array, encodeInto(v, dataView, offset) -> end offset, decode(bytes), decodeAt(dataView, offset) -> { value, offset } }. id is the fixed id or null. Also messages (the array of codecs) and byName. decode demands that the buffer is exactly one message (RangeError if bytes are left over; use decodeAt to read from the start of a longer buffer). A field with no default that is missing from the object throws TypeError: Vec2: the field 'y' is missing; a list over capacity throws RangeError: Flock.members: a list of 9 items does not fit: it holds at most 8 (it is never truncated).
  • pasture.d.ts: Codec<T, I>, each enum as an object and a union type, an interface per message and a <Name>Init (what encode takes: the fields with a default are optional).
  • pasture.schema.json: { schema, namespace, contractFingerprint, enums[], messages[{ name, id, fingerprint, wireFingerprint, wireCanonical, minSize, maxSize, variableSize, flatFields, fitsInputOrAct, canBeViewRow, fields[...] }] }. fitsInputOrAct means the maximum size is at most 255 bytes. canBeViewRow means 1 to 1024 flattened fields. A u64, an i64 and big defaults are JSON strings.
The JavaScript is codecs, nothing more. It encodes and decodes the bytes of your messages and rows. The SDK has no JavaScript session, prediction or tick-message decoder: the client runtime is the C++ aether_client library.

Running the tool

aether-schema pasture.aether --cpp out/pasture/Messages.h --js out/pasture.js --dts out/pasture.d.ts --json out/pasture.schema.json
ArgumentMeaning
<schema.aether>The file to read. Required. An unreadable file: nothere.aether:1:1: error: cannot read the schema file, exit 1.
--cpp <path>Write the C++ header here. The include path <stem>/Messages.h is by convention: name the directory after the file.
--js <path>Write the ES module.
--dts <path>Write the TypeScript declaration.
--json <path>Write the JSON description (it includes the contract fingerprint).
--max-message-bytes <n>Raise the biggest message a client may send past the default 4096 (see Size checks). It must be a positive number.

With no output flag the tool only checks the file. It writes a file only when its content would change, so a build does not rerun for nothing. Exit codes: 0 success, 1 a schema or file error, 2 a usage error (an unknown option is aether-schema: unknown option --bogus followed by the usage line). It is installed as bin/aether-schema.

With CMake

CMakeLists.txt
aether_schema(meadow_messages SCHEMA meadow.aether JS ${CMAKE_BINARY_DIR}/web/gen)
target_link_libraries(meadow_server PRIVATE meadow_messages aether)
target_link_libraries(meadow_client PRIVATE meadow_messages aether_client)
ArgumentMeaning
<target>The name of the INTERFACE library to create. It carries the include directory of the generated header, so #include <meadow/Messages.h> works, where meadow is the file name without its extension. A second target <target>_generate is part of ALL, so the JavaScript is produced even when no C++ target links the header.
SCHEMA <file>Required. A path relative to the current source directory, or absolute. aether_schema(t): SCHEMA <file.aether> is required and aether_schema(t): the schema ... does not exist are fatal errors.
JS <dir>Also write <dir>/<stem>.js, .d.ts and .schema.json.
MAX_MESSAGE_BYTES <n>Passed to the tool as --max-message-bytes.

The generated header is written to ${CMAKE_CURRENT_BINARY_DIR}/aether_schema/<target>/include/<stem>/Messages.h and regenerated when the schema or the tool changes. The tool is found as the target aether-schema when you build the SDK with your project, else as the cache variable AETHER_SCHEMA_EXECUTABLE, else on the PATH (find_program ... REQUIRED). An installed SDK has the function in lib/cmake/aether/AetherSchema.cmake. An unknown argument is a fatal error.

aether_game(<name>) belongs to the SDK's own source tree: it builds the example games under examples/<name>/, and when examples/<name>/<name>.aether exists it calls aether_schema(<name>_messages ...) for you and links the result into the game's server, client and test. You do not write aether_schema for those games. (The option TEST_IN_AETHER_TESTS only decides whether its test is also listed in the main test executable.) A project outside the SDK calls aether_schema itself.

The fingerprint

The tool writes one 64-bit number into the header, meadow::contract, and into the JS and JSON. It is a hash over the whole file. What changes it and what does not, checked by regenerating variants of pasture.aether:

EditChanges the fingerprint?
adding, removing or renaming a fieldyes
changing a field's type or size (f32 to f64, string<16> to string<20>)yes
changing an enum valueyes
changing or adding a fixed idyes
a commentno
reordering the messagesno
changing a defaultno
changing the namespaceno

The version check

Both sides are given the contract: server::Game({.contract = meadow::contract}) on the server and client::Session session(url, {.contract = meadow::contract}) on the client. The client sends its fingerprint (and the file name) in the connect query; over the GameNetworkingSockets transport it goes in the first "hello" message, which needs helloWaitMs set on the server's options. The server compares.

ServerClientResult
contractthe same contractjoins
nonenonejoins (a game with no schema works as it always did)
contracta different onerefused: client and server were built from different versions of meadow.aether (rebuild the client from the same file)
contractnonerefused: this server requires the contract meadow.aether; the client sent none (build the client from the same file and give its Session the contract)
nonea contractrefused: the client was built from meadow.aether but this server has no contract (give the Game `.contract = <namespace>::contract`)

A refused client is closed with code 4000 and the reason. The client's onClose(code, reason) receives it; the server prints one line to stderr (aether: a client was refused: ...) and nothing else is touched. The rule in one sentence: both sides give a contract and they match, or neither does.

Trap: a long reason arrives cut. A close reason is at most 123 bytes (the rest is cut), so on the client the longest refusals end in mid-word. The server's stderr line has the whole text.

Next, views: how a row becomes something a client sees, and who sees which rows. Views and visibility.

Reference

Every public item of this page, with its signature, its parameters, what it returns and when it throws. Namespaces: aether::ecs unless said otherwise; aether::String and aether::List are aliases of the ecs ones.

Entity

enum class Entity : std::uint64_t {};
inline constexpr Entity kNoEntity{~std::uint64_t{0}};
constexpr bool isSet(Entity e) noexcept;
std::string toString(Entity e);
std::ostream& operator<<(std::ostream&, Entity);       // also std::formatter<Entity>
ItemMeaning
EntityLow 32 bits: slot index. High 32 bits: generation. Generation 0 is never alive. No arithmetic, no conversion to bool; static_cast<std::uint64_t>(e) is the raw id.
kNoEntityAll bits set. The value that means "no entity" (for example Player::entity of a spectator).
isSet(e)false for kNoEntity and for Entity{}; true otherwise. Says nothing about whether the entity is alive: use Registry::alive(e).
toString(e)"slot:generation", for example "7:2"; "none" if not set. Printing with operator<< or std::format gives the same.

See also: The registry for create, destroy and alive.

String<N>

template <std::size_t N> class String {
public:
    static constexpr std::size_t capacity = N;
    String() = default;                                   // empty
    explicit String(std::string_view s);                  // throws if s.size() > N
    String& operator=(std::string_view s);                // throws if s.size() > N
    std::string_view view() const;   operator std::string_view() const;
    std::string str() const;
    std::size_t size() const;        // bytes before the first NUL
    bool empty() const;
    friend bool operator==(const String&, const String&);
    friend bool operator==(const String&, std::string_view);
};
ItemDetail
N1 to 65535 (ecs::String<N>: N is the size in bytes, from 1 to 65535). N bytes in a row, zero padded.
construction, assignmentFrom text of at most N bytes. Longer text throws std::invalid_argument (ecs::String<16>: the string has 20 bytes and does not fit (it is never truncated). Cut it before, or use a larger String) and leaves the value unchanged. Construction is explicit: String<16>("Dolly"); assignment is not: name = "Dolly".
view(), conversion, str()The content up to the first NUL (or all N bytes). str() copies into a std::string.
size(), empty()Content length; empty() is true when the first byte is NUL.
capacityThe constant N.
String<16> name{"Ayla"};  name = "Brin";  std::string_view v = name;  bool same = (name == "Brin");

See also: Strings; for a hand-built field, Row::set on a string field has the same rule.

List<T, N>

template <class T, std::size_t N> class List {
public:
    using value_type = T;   using Count = std::uint8_t /* N <= 255 */ or std::uint16_t;
    List() = default;   List(std::initializer_list<T>);
    std::size_t size() const;  bool empty() const;  bool full() const;  static constexpr std::size_t capacity();
    void push(const T& v);   void erase(std::size_t i);   void pop_back();   void clear();
    T& operator[](std::size_t i);  T& front();  T& back();            // const overloads too
    T* begin();  T* end();                                          // only the items in use
    friend bool operator==(const List&, const List&);
};
ItemDetail
N1 to 65535 (aether::List<T, N>: N is the capacity, from 1 to 65535). T: a field type, String, Entity or a plain struct; not another List.
push(v)Adds at the end. Full: std::length_error, list unchanged.
erase(i)Removes item i; later items move up; the freed slot is zero. i >= size(): std::out_of_range.
pop_back(), front(), back()Empty list: std::out_of_range.
operator[](i)Item i in 0..size()-1; else std::out_of_range (aether::List<T, 4>::operator[]: index 6 but the list has 2 items).
clear()Zeroes the used items and sets the count to 0.
initializer listMore than N items throws like push.
in a rowField name.count (Count) then name[0] to name[N-1].

See also: Lists.

Type, TypeBuilder, Prim, FieldInfo

class Type {
public:
    static TypeBuilder define(std::string name, std::uint16_t version = 1);
    std::string_view name() const;          std::uint16_t version() const;
    std::uint32_t blockLength() const;      std::uint32_t rowSize() const;
    const std::vector<FieldInfo>& fields() const;
    const FieldInfo* find(std::string_view name) const;     // nullptr if none
    const FieldInfo* key() const;                           // nullptr if none
    template <class T> Field<T> field(std::string_view name) const;
    template <class... Ts> std::tuple<Field<Ts>...> fields(NameOf<Ts>... names) const;
    std::uint64_t fingerprint() const;
    const detail::StoreInfo* store() const;                  // null for a type whose rows are packed
    friend bool operator==(const Type&, const Type&);
};
struct FieldInfo { std::string name; std::uint16_t tag; Prim prim; std::uint32_t offset, slot;
                   std::uint16_t width, sinceVersion; bool isKey; std::vector<std::int32_t> enumValues; };
MemberDetail
Type::define(name, version = 1)Starts a definition. name is the component's name and, if saved, the topic name. version is the type's version. Returns a TypeBuilder.
name(), version()As given.
blockLength()The size of a packed row in bytes (0 for a tag).
rowSize()The size of a row as the pool stores it: sizeof(S) for a component stored as a struct, blockLength() otherwise.
fields()The fields in declaration order. FieldInfo::offset is where the field is in a packed row (wire, view rows, records); slot is where it is in the stored row (equal to offset except for a struct-stored component); width in bytes; tag the durable tag; sinceVersion; isKey; enumValues (for Enum; empty means any value).
find(name), key()The field or nullptr.
field<T>(name)A handle. std::invalid_argument if there is no such field or T is not the C++ type of its kind; compile error if T is the C++ type of no kind. The T for each kind is in Field kinds.
fields<Ts...>(names...)Several handles, in the order of the names. The error is that of the first wrong one.
fingerprint()FNV-1a-64 over the name and, per field in tag order: tag, kind, since-version, key flag, name, enum values and string width. Equals rune::codec::fingerprint of the equivalent descriptor.
operator==True if both are the same definition, or have equal names, versions and fields.
const auto type = ecs::Type::define("Shepherd").uint64("id").key().string("name", 16).build();
const auto name = type.field<std::string_view>("name");

TypeBuilder

MethodDetail
int8 uint8 int16 uint16 int32 uint32 int64 uint64 float32 float64 boolean (name)Add a field of that kind. Returns the builder.
enumeration(name, allowed = {})An int32 with an optional list of allowed values.
string(name, bytes)A fixed string of bytes (a std::uint16_t). 0 makes build() throw.
key()Marks the last field as the key. See the rules.
tag(std::uint16_t)The durable tag of the last field (default: its position).
since(std::uint16_t)The version that added the last field (default 1).
build()Returns the Type. Throws std::invalid_argument naming the field: duplicate name, duplicate tag, a second key, a key of a kind that cannot be one, key()/tag()/since() before any field, a zero-length string. No fields is allowed: it is a tag.

Prim

enum class Prim : std::uint8_t { Int32 = 0, Int64 = 1, UInt16 = 2, UInt32 = 3, UInt64 = 4, Float64 = 5, Bool = 6,
                                 Enum = 7, FixedStr = 10, Int8 = 11, UInt8 = 12, Int16 = 13, Float32 = 14 };

See also: Kind 2, Saving for keys and fingerprints.

Field, Row, ConstRow

template <class T> struct Field { std::uint32_t offset; std::uint16_t width; /* debug: type tag */ };

class ConstRow {
public:
    explicit operator bool() const;                      // false: no row (absent, or a dead entity)
    const std::uint8_t* data() const;
    template <class T> T get(Field<T> f) const;
    template <class S> S get() const;                    // the whole row as the struct S
    template <class First, class... Rest> auto get(First first, Rest... rest) const;   // a path
};
class Row : public ConstRow {
public:
    std::uint8_t* data() const;
    template <class T> Row set(Field<T> f, std::type_identity_t<T> v) const;   // chains
    template <class S> Row set(const S& value) const;                         // the whole row
    template <class First, class... Rest> Row set(First first, Rest... rest) const; // a path, value last
};
CallDetail
get(Field<T>)Reads the field. A string field gives a std::string_view up to the first NUL (or all width bytes). A debug build aborts if the row is false or the field belongs to another type; a release build does not check.
set(Field<T>, v)Writes the field. A string longer than the field throws std::invalid_argument("ecs::Row::set: the string is longer than its field") and leaves the row unchanged; a shorter one is zero padded. Returns the row, so calls chain.
get<S>(), set(value)Read or write every member of the plain struct S (S must have the row's layout: the same members in the same order).
path formsrow.get(&Sheep::position, &Vec2::y), row.set(&Sheep::hp, 5). Member pointers go down; an integer goes into a std::array or List (std::out_of_range past the end). Compile errors name the problem (ecs: a path into a row is member pointers ... and indexes into a std::array or an aether::List).

A row is valid until an entity gains or loses that component (or, for a component in a group, another of the group). See also: The registry.

Typed<S> and registering

template <class S> Typed<S> Registry::component(std::size_t reserveRows = 1024);
template <class S> Typed<S> Registry::component(const PersistOptions&, std::size_t reserveRows = 1024);
template <class S, class Saved> Typed<S> Registry::component(const PersistWith<Saved>&, std::size_t reserveRows = 1024);
Component Registry::component(Type type, std::size_t reserveRows = 1024);
Component Registry::component(Type type, const PersistOptions&, std::size_t reserveRows = 1024);

template <class S> S& Registry::add(Entity e, const Typed<S>& c, const S& value);
template <class S> S& Registry::add(Entity e, const Typed<S>& c);
template <class S> S& Registry::get(Entity e, const Typed<S>& c);       // const overload too
template <class S> S* Registry::tryGet(Entity e, const Typed<S>& c);     // nullptr if absent
ItemDetail
component<S>(reserveRows)Registers the flat struct S under its name; returns a Typed<S> that converts to Component. reserveRows: rows to allocate up front. Compile errors: not trivially copyable, not flat (messages). Throws std::invalid_argument for a name clash with another struct or a hand-built component.
component<S>(persist(...))The saved forms; see Saving.
component(Type)Registers a hand-built type. Throws ecs::Registry: a component or view named 'x' exists for a taken name.
add(e, typed, value), add(e, typed)Adds a row (zeroed, or a copy of value) and returns S&. std::logic_error if the entity is dead, already has the row, or is in a running query of it. On a saved component add takes the key from the keyed member.
get(e, typed)S&. std::logic_error if the entity is dead or has no row (use has(e, handle) / tryGet(e, handle)).
tryGet(e, typed)S*, or nullptr.
Component-typed add/getregistry.add(e, component) returns a Row; registry.get(e, component) returns a Row, false if absent.
Trap. A comment in Typed.h shows registry.get(e, position).set(&Position::x, 3.0). With a Typed<Position> that call returns Position&, which has no set. Use the struct, or convert the handle: registry.get(e, ecs::Component(position)).set(...).

Local components

template <class T> Local<T> Registry::local();
template <class T> T& Registry::add(Entity e, const Local<T>& c, T value);
template <class T> T& Registry::add(Entity e, const Local<T>& c);                 // default object
template <class T> T& Registry::get(Entity e, const Local<T>& c);                 // const overload too
template <class T> T* Registry::tryGet(Entity e, const Local<T>& c);               // const overload too
template <class T> void Registry::deferAdd(Entity e, const Local<T>& c, T value);
template <class T> void Registry::onAdd(const Local<T>& c, std::function<void(Entity, T&)> fn);
template <class T> void Registry::onRemove(const Local<T>& c, std::function<void(Entity, T&)> fn);

template <class T> class Local { public: operator Component() const; Component component() const; };
// server::Game
template <class T> ecs::Local<T> local();
ItemDetail
local<T>()Registers T (or returns the handle it already has: matched by std::type_info). Compile errors: T's move constructor or destructor is not noexcept; not movable; alignment above 64; not a plain class type. Throws std::invalid_argument if the name is taken by a component or view. local<T>(ecs::persist(...)) is a compile error.
add(e, c, value)Moves value in; returns the object. std::logic_error if the entity is dead, already has it, or is in a running query of it.
add(e, c)A default-constructed object. Compile error if T has no default constructor.
get, tryGetThe object; get throws std::logic_error when absent, tryGet returns nullptr. Valid until the next structural change of this component.
deferAdd(e, c, value)Adds when the outermost running query ends (or at once outside a query). The value is kept in the registry's own block storage until then. A deferred add that never runs destroys the value.
onAdd, onRemoveHooks that get the object as T& (even onRemove). The hook rules are on the registry page.
Query argumentslocal gives T&; ecs::read(local) const T&; ecs::optional(local) T*; ecs::optional(ecs::read(local)) const T*; ecs::none(local) / ecs::any(local) filter.

See also: Kind 3.

Function views

template <class T, class Fn> ViewBuilder& ViewBuilder::from(const Local<T>& source, Fn fn);
template <class C, class Fn> ViewBuilder& ViewBuilder::from(const Typed<C>& source, Fn fn);
ViewBuilder& ViewBuilder::fromFunction(Component source, Type rowType, std::string rowName,
                                       std::function<bool(const std::uint8_t*, std::uint8_t*)> fn);
// Fn:   void(const Source& in, Out& out)    or    bool(const Source& in, Out& out)
ParameterDetail
sourceA Local<T> (the function takes const T&) or a Typed<C> (const C&).
fnA lambda with explicit parameter types, a functor, or a plain function (also noexcept). Returns void, or bool where false means "no row this commit". The row type Out is the second parameter, a plain flat struct taken by non-const lvalue reference; the view is named after it (view<Out>(name) on a Game, View::define<Out>(name)).
fromFunctionThe byte-level form: fn(sourceRow, outRow) fills the zeroed row of rowType (the view's fields, in order) and returns false to skip it. rowName names the struct for messages.

Compile errors, verbatim: view.from(source, function): the function takes (const Source& in, Out& out) where Out is the struct of the view (a plain struct), taken by non-const reference; view.from(source, function): the function returns void, or bool (false: this entity has no row); view.from(local, function): the function takes (const T& in, Out& out) where T is the local component's type; and for a generic lambda view.from(source, function): pass a lambda with explicit parameter types, [](const Source& in, Out& out) { ... }, or a plain function void f(const Source&, Out&) (returning bool to skip a row). A generic lambda (auto parameters) or an overloaded function name cannot be inspected: write the parameter types out. Using a local with from(local) alone, join, leftJoin, select, selectAll, groupBy, sum, avg, min, max, last or aggregate is: a local component stays on the server; use view.from(component, function) to show part of it (select) (the last word names the call).

See also: The bridge, Views and visibility.

server::Game row access

template <class... Rows> ecs::Entity spawn(Rows&&... rows);
template <class Row, Flag... Fs> Row& get(ecs::Entity e, Flags<Fs...> flags = {});     // const Row& with READONLY
template <class Row> void set(ecs::Entity e, const Row& row);
template <class T> ecs::Local<T> local();
template <class Row> ecs::Component componentOf();
static constexpr auto READONLY = aether::READONLY;
ItemDetail
spawn(rows...)One entity with one row per argument. Flat struct: a component. Other type: a local component. A saved row is added with its key (empty, zero, too long or taken: throws, no entity left behind). Returns the entity.
get<Row>(e)The row as Row& (view row, local, or saved). std::logic_error, in every build, if the entity is dead or has no such row (messages above).
get<Row>(e, READONLY)const Row&. Never marks a saved row as touched.
set(e, row)Assigns a whole row; same errors. A local needs a copy-assignable type (compile error otherwise).
local<T>()As Registry::local, plus the noexcept check with its own message.
componentOf<Row>()The Component of a flat struct, registered if need be. Finishes a pending persist<Row>() first.
READONLYA flag value (aether::READONLY) that changes the return type of get to const. READONLY | OTHER is the way further flags would combine.

See also: The server for onJoin, players(), snap.

Macros

MacroArgumentsEffect and limits
AETHER_COMPONENT(S, m1, m2, ...) each m is a member name or key(name)Describes S by that list (1 to 32 members). S must be a plain aggregate.
AETHER_STRUCT(S, m...)As above, for a part.
AETHER_VIEW(S, "name", m...)Plus the view's name.
AETHER_INPUT(S, m...)Plus: the held input.
AETHER_ACTION(S, id, m...)id 1 to 255. Members optional.
AETHER_MESSAGE(S, type, m...)type 16 to 255. Members optional.
AETHER_COMPONENT_EMPTY(S)A struct with no members.

See also: The optional macros.

aether-schema and aether_schema()

aether-schema <schema.aether> [--cpp <Messages.h>] [--js <out.js>] [--dts <out.d.ts>] [--json <out.schema.json>] [--max-message-bytes <n>]
aether_schema(<target> SCHEMA <file.aether> [JS <dir>] [MAX_MESSAGE_BYTES <n>])

Every flag and argument is explained in Running the tool. Exit codes: 0 ok, 1 schema or file error, 2 usage.

Contract

struct aether::Contract {
    const char* schema = "";           // the file name, "meadow.aether": what a refusal says
    std::uint64_t fingerprint = 0;     // the tool's contractFingerprint of the file
    constexpr explicit operator bool() const;     // fingerprint != 0
    std::string query() const;          // "contract=<16 hex digits>&schema=<file name>"
};
server::GameOptions::contract;          // Contract, default none
client::SessionOptions::contract;       // Contract, default none
ItemDetail
schemaThe name shown in a refusal.
fingerprintZero means "no contract". The generated meadow::contract is Contract{"meadow.aether", 0x776d18f6a0554fa1}.
query()The text sent in the connect query. The file name is left out if it has a character a query cannot hold (it keeps letters, digits, ., _, -).
RefusalClose code 4000 with the reasons in the table.