ARCANUM
Guide ยท testing and building

Testing and building

A game test starts your World on a real port, connects real client sessions to it and checks what they see, all in one process. Then the other half: the libraries, the CMake functions and the commands that build it all.

A first test

Aether does not give you a fake server to test against. <aether/testing.h> starts your real World (a server::Game) on a free port, on a thread of its own, and your test connects real client::Sessions to it. What the test sees is what a player would see, including prediction, snaps, saving and the rate limit. You need the server library aether and, for the client half, aether_client (the first links the second).

Here is the smallest useful test for the meadow world (the running example): two players connect, and the server counts them. It uses googletest, as the SDK's own tests do.

tests/MeadowTest.cpp
#include <gtest/gtest.h>

#include <aether/testing.h>

#include "meadow/World.h"

using namespace std::chrono_literals;

TEST(Meadow, TwoPlayersJoin) {
    ecs::MemoryStore store;
    test::Server<World> server(store);                      // World's constructor argument: the store
    client::Session ayla(server.url("name=Ayla"), {.contract = meadow::contract});
    client::Session bo(server.url("name=Bo"), {.contract = meadow::contract});
    ayla.connect();
    bo.connect();
    ASSERT_TRUE(test::pump(ayla, bo, [&] { return ayla.ready() && bo.ready(); }));
    EXPECT_EQ(server.run([](World& w) { return w.players().size(); }), 2u);
}

Notice four things.

  • test::Server<World> server(store) constructs the World with the arguments you pass (here the store its constructor takes), starts listening on a free port and returns once it is listening. The destructor stops it.
  • server.url("name=Ayla") is the address to give a Session: a ws:// address on the loopback interface and the bound port, ending /?name=Ayla. The query string is what p.query("name") reads in onJoin.
  • The server runs on its own thread, so nothing happens on the client side until you poll the sessions. test::pump(ayla, bo, until) polls both until your function returns true, or gives up after 10 seconds and returns false.
  • server.run(fn) runs fn(World&) on the world thread, between two ticks, and hands you what it returns. It is how you look at the server's state without racing it.

Time is the real clock: tests take real seconds, and a test that has to wait for a state waits for it. The server also ticks faster than a production one, which matters: see Config.

Polling: pump, pumpFor and waitFor

A client session does nothing by itself: session.poll() reads the connection and applies what arrived. The three helpers differ in what they do while they wait.

HelperWhat it doesPolls the sessions?Returns
pump(session..., until [, timeout])polls every session, checks until() before each round, until it is true or the time is upyestrue if until became true; at the end it checks once more
pumpFor(duration, session..., [each])polls every session for duration; calls each() after every round of polls if the last argument is a functionyesnothing
waitFor(pred [, timeout])checks pred() every 2 msnotrue if pred became true

pump

ASSERT_TRUE(test::pump(a, b, [&] { return a.ready() && b.ready(); }));      // 10 s
ASSERT_TRUE(test::pump(me_session, [&] { return me.now().x > 100; }, 60s));    // 60 s: sessions, the function, then the timeout

The arguments are one or more sessions, then the function (returning bool), then optionally a timeout as a std::chrono::duration. The default is 10 seconds; a real lap in a debug build may need more. Fewer than two arguments (a session and a function) is a compile error. The function must return bool.

pumpFor

Use it to let time pass while something keeps happening on the client, such as holding a key down. The last argument is called after each round of polls:

test::pumpFor(500ms, s, [&] { me.input(Move{1, 0}); });    // hold "right" for half a second
test::pumpFor(300ms, a, b);                                 // just let both sessions settle

waitFor

waitFor is for conditions on the server side: it sleeps in 2 ms steps and polls nothing. It takes a std::chrono::milliseconds timeout (default 10 s) or a plain int of milliseconds.

ASSERT_TRUE(test::waitFor([&] { return server.run([](World& w) { return w.players().empty(); }); }));
EXPECT_FALSE(test::waitFor([] { return false; }, 300ms));
Trap. waitFor proves the server has some state, never that a client has seen it. And it does not poll, so a predicate on a client's view (a.ready(), a view's contents) can never become true inside it: the client is not reading anything. Verified: waitFor([&]{ return a.ready(); }, 300ms) on a connecting session returned false, and the same predicate in pump returned true. For anything a client must see, use pump until the client's own view shows it.

Waiting for the server to catch up with your input

A predicted client numbers its inputs. The server tells the client the last one it applied, so me.acked() reaching the number you took means "the server has applied what I sent up to here". Compare with a number you took earlier, never with me.sequence(): the client keeps cutting steps, so acked() and sequence() are never equal for long.

TEST(Meadow, AHeldMoveWalksTheWalker) {
    ecs::MemoryStore store;
    test::Server<World> server(store);
    client::Session s(server.url("name=Ayla"), {.contract = meadow::contract});
    auto me = s.predict<Walker>();
    me.apply<Move>(step);
    s.connect();
    ASSERT_TRUE(test::pump(s, [&] { return me.active(); }));

    test::pumpFor(500ms, s, [&] { me.input(Move{1, 0}); });
    EXPECT_GT(me.now().x, 5);                                // predicted: the client ran step() itself

    const std::uint32_t target = me.sequence();             // the last step cut so far
    ASSERT_TRUE(test::pump(s, [&] { me.input(Move{0, 0}); return me.acked() >= target; }));
    const Walker onServer = server.run([](World& w) { return w.get<Walker>(w.players().begin()->entity); });
    EXPECT_GT(onServer.x, 5);                                // the server ran the same rule
}

Notice that me.input(Move{0, 0}) sits inside the predicate: input() only says what is held, and the numbered steps are cut by the polls. Holding nothing keeps the stream going so the ack keeps moving.

Snaps are visible for one poll

view.snapped(e) is true only in the poll that brought the snap. Check it inside the predicate, and remember it:

TEST(Meadow, ASnapIsSeenInTheSamePollItArrived) {
    ecs::MemoryStore store;
    test::Server<World> server(store);
    client::Session s(server.url("name=Ayla"), {.contract = meadow::contract});
    auto walkers = s.view<Walker>();
    auto me = s.predict<Walker>();
    me.apply<Move>(step);
    s.connect();
    ASSERT_TRUE(test::pump(s, [&] { return me.active(); }));
    server.run([](World& w) { for (auto& p : w.players()) w.snap(p.entity, Walker{50, 0, 60}); });
    bool sawSnap = false;
    ASSERT_TRUE(test::pump(s, [&] {
        me.input(Move{0, 0});
        if (walkers.snapped(s.me())) sawSnap = true;         // ask inside the predicate: it is true only in the poll that brought it
        return sawSnap;
    }));
    EXPECT_NEAR(me.now().x, 50, 1.0);
}

Next: more of what server.run and the server object give you.

Looking at and changing the server

server.world is the World itself. Touch it directly only before the first client connects. After that the world thread owns it, and every look at it must go through server.run:

const auto seats = server.run([](World& w) { return w.players().size(); });          // read
server.run([](World& w) { for (auto& p : w.players()) w.get<meadow::Purse>(p.entity).gold += 5; });   // change

run posts the function to the world (World::post) and waits for the answer. The function runs on the world thread, between two ticks, so the Registry's one-thread rule holds. It returns whatever the function returns (a copy: take values out, do not return references into the registry). If the world thread does not run it within 10 seconds it throws std::runtime_error: test::Server::run: the world thread did not run it in 10 s (is the server stopped?) (verified after server.stop()).

The world may be a server::Game or a server::Host: anything with post(fn). For a Host, the games are members of your own Host subclass, and a client chooses one with ?game=:

struct Both : server::Host { Walker walker; Duel duel; Both() { add("walker", walker).add("duel", duel); } };
test::Server<Both> server;   client::Session a(server.url("game=duel"));

server.stop() stops the server early (the destructor does the same, and calling it twice is harmless). server.port() is the bound port. A test of a restart destroys one test::Server and builds a second on the same store: see Saving and restarting.

More tests for meadow

Chat reaches everyone

TEST(Meadow, ChatReachesEveryone) {
    ecs::MemoryStore store;
    test::Server<World> server(store);
    client::Session a(server.url("name=Ayla"), {.contract = meadow::contract});
    client::Session b(server.url("name=Bo"), {.contract = meadow::contract});
    std::vector<std::string> heard;
    b.on<meadow::Said>([&](const meadow::Said& m) { heard.push_back(m.who.str() + ": " + m.text.str()); });
    a.connect();
    b.connect();
    ASSERT_TRUE(test::pump(a, b, [&] { return a.ready() && b.ready(); }));
    a.send(meadow::Chat{String<80>("hello")});
    ASSERT_TRUE(test::pump(a, b, [&] { return !heard.empty(); }));
    EXPECT_EQ(heard[0], "Ayla: hello");
}

Register the message handler (b.on<Said>(...)) before connect(), so no message is missed.

Saving and restarting

To test that something is saved, give the World a store you own, make the first server go out of scope (stopping it flushes what is dirty to the store) and start a second one on the same store.

TEST(Meadow, GoldSurvivesARestart) {
    ecs::MemoryStore store;                                  // outlives both servers
    {
        test::Server<World> server(store);
        client::Session s(server.url("name=Ayla"), {.contract = meadow::contract});
        auto purses = s.view<meadow::Purse>();
        s.connect();
        ASSERT_TRUE(test::pump(s, [&] { const auto p = purses.find(s.me()); return p && p->gold >= 5; }));
        s.close();
        ASSERT_TRUE(test::waitFor([&] { return server.run([](World& w) { return w.players().empty(); }); }));
    }                                                        // the server stops here, and saves
    EXPECT_GE(store.saves(), 1u);

    test::Server<World> again(store);                        // a second server on the same store
    client::Session s(again.url("name=Ayla"), {.contract = meadow::contract});
    auto purses = s.view<meadow::Purse>();
    s.connect();
    ASSERT_TRUE(test::pump(s, [&] { return purses.find(s.me()).has_value(); }));
    EXPECT_GE(purses.find(s.me())->gold, 5u);                // her gold came back
}

Notice that the test waits for the player to be gone (waitFor on the server side) before leaving the scope: a leaving player is saved at once, and the rest at shutdown. For a restart that goes through a file, use ecs::FileStore store(path) in the same way (tests/FarmTest.cpp does).

Counting saves

MemoryStore::saves() counts rows put (a new row or a changed one) and erases() rows removed, since the store was made. Reading does not count, and neither does a FileStore reading its file when it opens. That is how you prove "nothing was saved":

const auto before = store.saves();
// ... a player joins, looks, leaves, changing nothing ...
EXPECT_EQ(store.saves(), before);

A save is conflated: with the default window of about a second a changed row is saved once per window, so waitFor([&]{ return store.saves() == n; }) is the right form and a bare EXPECT_EQ right after the change is not. MemoryStore::size(topic) says how many keys a topic holds.

The rate limiter is on in tests too

A client may send 30 messages a second (a burst of 60) and 16 KiB a second. A test that sends faster is closed. Verified: 200 chat messages sent at once close the session with code 4290, after a warning to onWarning: too many messages: the limit is 30 messages a second (burst 60), 16384 bytes a second; what is over it is dropped, and the close reason is closed for sending too much: the limit is 30 messages a second (burst 60), 16384 bytes a second. Turn the limit off for such a test with {.perSecond = 0} (the byte limit and the violation rules go with it):

TEST(Meadow, AFloodNeedsNoRateLimit) {
    ecs::MemoryStore store;
    test::Server<World> server(test::Config{.rate = {.perSecond = 0}}, store);
    client::Session s(server.url("name=Ayla"), {.contract = meadow::contract});
    int heard = 0;
    s.on<meadow::Said>([&](const meadow::Said&) { ++heard; });
    s.connect();
    ASSERT_TRUE(test::pump(s, [&] { return s.ready(); }));
    for (int i = 0; i < 100; ++i) s.send(meadow::Chat{String<80>("spam")});
    EXPECT_TRUE(test::pump(s, [&] { return heard == 100; }));
}

TEST(Meadow, TheDefaultLimitClosesAFlooder) {
    ecs::MemoryStore store;
    test::Server<World> server(store);
    client::Session s(server.url("name=Ayla"), {.contract = meadow::contract});
    std::uint16_t code = 0;
    s.onClose([&](std::uint16_t c, std::string_view) { code = c; });
    s.connect();
    ASSERT_TRUE(test::pump(s, [&] { return s.ready(); }));
    for (int i = 0; i < 200; ++i) s.send(meadow::Chat{String<80>("spam")});
    ASSERT_TRUE(test::pump(s, [&] { return code != 0; }));
    EXPECT_EQ(code, 4290);
}

The rate limiter counts messages, not held inputs: the ping echo and the held-input message are not counted.

Config: the server a test starts

The default test server is not your server. test::Config has the settings, in this order (designated initialisers must follow it):

test::Server<World> server(test::Config{.tickRate = 20}, store);       // the rate your server.cpp gives the App
FieldDefaultMeaning
tickRate50ticks a second. Not the 20 of a usual server.cpp: tests are quick, but a test that counts ticks or reads session.tickRate() wants .tickRate = 20. A Host's games tick at this rate divided by a whole number.
flushRate200how often the network thread writes queued messages (Options::flushRate).
rateserver::RateLimit{}what a client may send: the server default of 30 messages a second, burst 60, 16 KiB a second. {.perSecond = 0} for no limit.
stalledSeconds300Options::stalledSeconds: a connection whose output is blocked this long is closed (0 = never).
linkReportIntervalMs250Options::linkReportIntervalMs: how often the world gets each client's byte rates and backlog (0 = no link reports).
transportnullptranother transport than WebSocket (see below). url() then gives gns://....
gameSocket{}Options::gameSocket (fake loss and lag, ...): only read with a transport that is a GameSocket.

The signal handlers are off in a test server (handleSignals = false). The constructor forms are: Server(), Server(args...) (World's constructor arguments; the first must not be a Config) and Server(Config, args...).

Another transport

server::GameSocket gs{0};                                   // GameSocketServer.h, link aether_gns; only has to live through the constructor
test::Server<World> server(test::Config{.transport = &gs}, store);
client::Session s(server.url());                            // a gns:// address: include aether/client/GameSocket.h, link aether_client_gns

Only one GameSocket server may run at a time in a process.

When the server does not start

The constructor waits up to 10 seconds for the server to listen, and throws std::runtime_error("test::Server: the server did not start listening in 10 s") if it does not. A port that cannot be bound throws test::Server: the world's onStart threw or no port could be bound (see stderr). A world whose onStart throws prints aether: exception on the world thread: onStart threw, so the world did not start: <what> (fix the cause; start() does not retry) on stderr. In a trial the constructor still returned with a port in that case (the sessions then never become ready), so watch stderr first when a test hangs at ready().

Bots: headless clients

A bot is a client that does what a game client does, minus the drawing. Because a test drives it from one thread (pump polls it), a bot is just a class that owns a client::Session, the views and the prediction a client needs, and exposes the verbs and what it saw. Declare the Session first: the views and the prediction are made from it.

tests/MeadowBot.h
#pragma once
#include <aether/client.h>

#include "meadow/Shared.h"

using namespace aether;

// A headless meadow client: it does what a game client does, minus the drawing.
class Bot {
public:
    client::Session session;                          // declared first: the others are made from it
    client::TypedView<Walker> walkers;
    client::TypedView<meadow::Purse> purses;
    client::Prediction<Walker> me;
    std::vector<std::string> heard;                   // what the server said

    Bot(const std::string& url, const std::string& name)
        : session(url + "?name=" + name, {.contract = meadow::contract}),
          walkers(session.view<Walker>()),
          purses(session.view<meadow::Purse>()),
          me(session.predict<Walker>()) {
        me.apply<Move>(step);                         // the same rule the server runs
        session.on<meadow::Said>([this](const meadow::Said& s) { heard.push_back(s.who.str() + ": " + s.text.str()); });
        session.connect();
    }

    bool ready() const { return session.ready() && me.active(); }
    void walk(float dx, float dy) { me.input(Move{dx, dy}); }
    void say(const char* text) { session.send(meadow::Chat{String<80>(text)}); }
    std::uint32_t gold() const {
        const auto purse = purses.find(session.me());
        return purse ? purse->gold : 0;
    }
};
TEST(MeadowBots, TwoBotsMeetAndTalk) {
    ecs::MemoryStore store;
    test::Server<World> server(store);
    Bot ayla(server.url(), "Ayla"), bo(server.url(), "Bo");
    ASSERT_TRUE(test::pump(ayla.session, bo.session, [&] { return ayla.ready() && bo.ready(); }));
    EXPECT_EQ(bo.walkers.size(), 2u);                       // both are within the default radius of 64
    ayla.say("hello");
    ASSERT_TRUE(test::pump(ayla.session, bo.session, [&] { return !bo.heard.empty(); }));
    EXPECT_EQ(bo.heard[0], "Ayla: hello");
    test::pumpFor(300ms, ayla.session, bo.session, [&] { ayla.walk(1, 0); });
    EXPECT_GT(ayla.me.now().x, 1);
    EXPECT_GT(ayla.gold(), 0u);                              // a coin a tick, seen by its owner only
    EXPECT_EQ(bo.purses.size(), 1u);                         // Bo sees only its own purse
}

The SDK's example games each ship such a class (examples/raid/Bot.h, examples/kart/Bot.h, examples/arpg/Bot.h) with a frame(dt) method that computes what the client would draw and a dump() that renders what it sees as text, so a failing test can print it. tests/support/NetBots.h has a bot that samples the age of its rows, drive / driveFor to poll many bots, and settle to stop the world and wait until every bot holds exactly what the server has; the network tests use them with many bots at once.

Tip. Name the gtest suite like the game (TEST(Meadow, ...)) so ctest -R Meadow runs it. Put the part of a game that tells a player what happened into a dump() on the bot: a failing EXPECT can print it with << bot.dump().

Pitfalls, in one place

  • The server ticks at 50 a second, not 20. Pass .tickRate = 20 if the test counts ticks or reads the tick rate.
  • waitFor does not poll the clients and proves only server state. Use pump for what a client sees.
  • pump gives up after 10 s and returns false: always ASSERT_TRUE it, or the rest of the test runs against a client that never connected. Give a longer timeout as the last argument.
  • Check snapped() inside the predicate: it is valid only in the poll that brought the snap.
  • Compare acked() with a number you took, never with sequence().
  • Touch the World only through server.run once a client has connected.
  • Declare the Session first in a class that holds views and predictions.
  • The rate limiter is on. A test that floods needs {.rate = {.perSecond = 0}}.
  • Designated initialisers must follow the declared order of test::Config, GameOptions, InputPolicy and Options.
  • One GameSocket server per process; a test file that mixes them needs its own executable (the SDK's GNS tests are separate executables for that reason).
  • The libraries and your code must agree on NDEBUG. Row, ConstRow and Field carry a type tag in debug builds and nothing in release builds (aether/ecs/Type.h), so a program compiled with -DNDEBUG against a library built without it has a different layout: in a trial it crashed inside Registry::add.

Shaping a link for measurements

Everything above runs on the loopback interface, which never loses a packet. To see what the SDK's link figures (Player::link()), budgets and prediction do on a slow or lossy path, the repository has tests/netns/link.sh: it runs a command across a virtual cable that the kernel shapes (tc netem for delay, jitter, loss and reordering, tc tbf for a rate cap). It needs no root and no ip netns add: it makes a user and network namespace with unshare -r -n. It is for measurements you run by hand, alone on the machine; for the test suite, the in-process UDP relay (tests/support/UdpRelay.h) and the TCP relay (tests/support/NetProxy.h) play that part.

One-time setup

Both last only until a reboot. link.sh checks them first, prints these two lines and exits with status 77 if one is missing:

sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
sudo modprobe -a sch_netem sch_tbf veth

Use

tests/netns/link.sh [conditions] [--at TIME key=value ...] -- <command...>
ConditionUnitMeaning
--ratebytes/scap on the way to the client (tbf, burst 1600); 0 = no tbf, netem alone
--uplink-ratebytes/sthe same, on the way to the server
--delay, --jittermseach way (applied on both ends)
--losspercentrandom loss to the client (the end where the command runs)
--loss-uppercentrandom loss to the server
--queuebytestbf limit (default: max(8192, rate/4))
--reorderpercentnetem reordering on both ends; needs --delay above 0
--aqmfq_codel under tbf instead of the plain byte limit

Conditions can change while the command runs: --at 10s rate=16000 --at 20s rate=0 (times like 10s or 500ms; the keys are the condition names without --). AETHER_LINK_VERBOSE=1 prints tc qdisc show at the start and before and after each change.

The command runs in the outer namespace, on one end of a small point-to-point link; the other end is in a child namespace held by a sleeping process. The command gets two environment variables: AETHER_NETNS_SERVER_ADDR, the address to bind a server to, and AETHER_NETNS_PEER, /proc/<pid>/ns/net of the peer namespace: setns into it around a client's connect to put the client's socket on the far side of the shaped link. When the command exits, one line is printed for the driver to parse:

AETHER_NETNS_DROPS to_client_queue=N to_client_netem=N to_server_queue=N to_server_netem=N

(queue is tbf or fq_codel drops, netem is random loss.) The script exits with the command's status. Each end has netem as its root queue (delay, jitter, loss, reorder) and tbf as its child, so packets waiting out their delay do not count against the queue; netem may reorder packets when --jitter exceeds the gap between them.

Check the setup once

tests/netns/link.sh --rate 16000 --delay 25 -- ping -c 5 -i 0.2 <the peer address>

The average round trip must read 45 to 70 ms (2 x 25 ms delay plus about 5 ms of serialisation of an 84-byte packet at 16,000 B/s; the first reply is about 100 ms because of ARP). With --rate 0 --delay 0 it reads under 5 ms. The peer address is printed in tests/netns/README.md.

Baselines and scenarios

tests/netns/baseline.sh <path to link_measure> [seeds] [condition,...] (from the repository root; seeds default 5) runs link_measure --rows ws,gns through link.sh for six conditions (128 kbit/s, 512 kbit/s and 2 Mbit/s, each at 50 ms and 200 ms round trip) and prints, each line prefixed with the condition's name, the program's median lines and link.sh's drops line. tests/netns/scenarios.sh <path to link_measure> [seeds] [cell,...] runs WebSocket and GameNetworkingSockets side by side over a fixed list of named cells and prints CELL <name> ok or CELL <name> FAIL ... and a last line SCENARIOS <failed> failed of <n>; it exits 1 if a cell failed and 77 if the link cannot be made. Both need link_measure, built with the benchmarks (below) and the GNS flavour.

The benchmarks

Benchmarks are off by default. Configure with -DAETHER_BUILD_BENCHMARKS=ON to build them. They are measurements, not tests, and they live in benchmark/. This page gives names and what each measures; the numbers belong to the machine, the build type and the arguments they were run with, so run them on your own.

TargetWhat it measuresNeeds
flush_benchmarkend-to-end throughput of the echo path with the WebSocket flavour: N real clients send small messages in bursts to an echo world and wait for the echoes; prints messages a second. Arguments: [flushRate=30] [clients=50] [messagesPerClient=400] [burst=8] [payloadBytes=32]
ecs_benchmarkthe cost of building a join view at commit, and of sending it to viewers through Interest. Arguments: [entities=10000] [ticks=300] [changedPercent=30]
ecs_persistence_benchmarkthe cost of persistence: collect over persisted rows of 64 bytes, and the memory it adds per row. Argument: [reps=30]
ecs_rune_benchmarkthe cost of the rune adapter (RuneStore) against a real broker, one thread, 64-byte rows. Arguments: [rows=10000] [ticks=300]AETHER_WITH_RUNE, AETHER_RUNE_BROKER=/path/to/rune_broker
local_benchthe add, erase, deferred add and iteration paths of flat and local components (best of 5 runs). Arguments: [flat=1000000] [local=200000]. Compiles the ecs sources itself at -O2 -DNDEBUG, so the numbers do not depend on the library's build type.
struct_benchthe cost of reading and writing a struct component and of the commit that builds view rows from it. Arguments: [rows=1000000] [repeats=7]. Compiled at -O2 -DNDEBUG whatever the build type.
client_poll_benchwhether the world thread needs to be told which clients have new input, or should read every client's input ring each tick (three ways, on the same traffic). Arguments: [ticks=300] [repeats=5]
ecs_iteration_benchmarkraw iteration, structural change and random access of the ECS against EnTT, with checksums that must agree (a line that differs is marked MISMATCH). Arguments: [passes=0 (auto)] [only]EnTT (conan install -o '&:with_benchmarks=True'); skipped with a message if absent
crowd_benchN players in one square, the server at 10 Hz, a server::Game with Jolt physics: what each client is sent under radius and byte budget. Arguments such as --players --square --rate --seconds --radius --budget-DAETHER_BENCH_JOLT=ON and the Jolt 5.6.0 package
fight_benchN players in one arena at 30 Hz, each a set of 16 kinematic Jolt bodies moved every tick from a scripted pose, with hit tests. Arguments such as --players --rate --seconds --parts --hits-DAETHER_BENCH_JOLT=ON and Jolt
link_measurehow the link figures and the client's view behave on a slow link, over WebSocket (through the TCP relay) and GNS (through the UDP relay), one client among 1,000 moving mobs, with .budget = headroom(); prints one line per (transport, link). Also driven by tests/netns/*.sh on a kernel-shaped link.GNS flavour
gns_probethree small measurements of raw GameNetworkingSockets with no SDK involved (live send-rate change, status fields on a shaped link, cost of setting rates on thousands of connections)GNS flavour

The scripts benchmark/run_budget.sh, run_crowd.sh, run_fight.sh and run_sendperf.sh run the crowd and fight matrices one run at a time, each inside a memory-limited systemd-run scope (<build dir> <results dir> [extra args]). LocalBench and StructBench report what they measured on the machine you run them on.

Diagnostics: what Aether prints and what you can ask

Messages on stderr

Everything the SDK says on its own starts with aether: . Here is each message and what it means.

MessageMeaning
aether: exception on the world thread: <what>a tick threw (a system, a handler outside the ones below). The world carries on. If it is onStart threw, so the world did not start: ... (fix the cause; start() does not retry) the world never ticks. Default of Options::onError.
aether: exception on the network thread: <what>the same on the network thread.
aether: a client was refused: <reason>onJoin threw, the contract did not match, or a load failed. The client is told the reason (close code 4000); what onJoin spawned is removed.
aether: the handler of message type N (client C) threw: ...a message handler threw; that message is lost, the tick carries on. Replace with Messages::onError.
aether: an input handler (client C) threw: ...the same for an input or act handler; replace with Inputs::onError.
aether: load<Row>('key') failed: ... (its callback was not called; give load a third argument, [](const std::string& why) { ... }, to handle it)a read of the store failed and no failure handler was given. A failed read is never reported as "absent".
aether: a function given to load<Row>(key, ..) threw and was dropped: ...the callback of load threw.
aether: the store did not save N row(s) of 'X' (...); they are tried again, aether: the store could not save: ...the store refused a batch or timed out. The same line again within 10 s is only counted: (N more like it) follows when it changes.
aether: the store did not confirm N row(s) within X ms at shutdown, they may be lost, aether: the store could not be closed: ...at shutdown the store did not acknowledge in time (flushTimeout()).
aether: RuneStore: connecting to ... failed (attempt N: ...); trying again for up to N s, aether: RuneStore: connected to ... at attempt Nthe rune broker is not reachable yet.
aether: ecs::View 'name': the function threw for entity 3:1 (...); its row is left out of the view until it stops throwinga function view threw. Goes to registry.onError if set, else stderr; once until it works again.
aether: the radius rule of client N (addr) answered X: a radius is a number >= 0 (a release build would use 0 and count it in stats().badRadius)your radius rule returned NaN or a negative number. Debug builds stop here.
aether: view 'v': A of its B rows are farther than R world units from a client's player, so that client is not sent them. R is the DEFAULT radius. To see farther: server::GameOptions{.radius = ...} (or InterestPolicy::radius). Printed once, in debug builds only.the most common reason "my entity does not show up": the default radius is 64.
aether: a budget AND a weight are set on a view with a radius: ... To make one entity matter, call p.focus(entity). Printed once.the weight does not apply to the rows around a player under a budget (they go by grid cell).
aether: ecs::Row::set: the field belongs to component 'b' but the row is a 'a' row. ... and ... on a false Row: the entity is dead or does not have the component. ...debug builds only: a Field used on a row of another component, or a read or write on a false row. The process aborts.

On the client

session.onError(fn(std::string_view)) gets connection errors and malformed server messages; without it they go to stderr as aether: ... for the mistakes of the game that must not pass silently. The ones you will meet: the server has no view called 'X' (it has: a, b), so session.view<Row>("X") stays empty. The view is looked up by the name the server gave it ...; malformed message of type N: ...; message of type N has the wrong size (B bytes); send(): the queue of messages waiting for the connection is full (256). Call connect() / reconnect() and poll(); send(): the connection's queue is full, the message was dropped; and the message X was not sent: the server takes X only as a predicted move .... session.onWarning(fn(text)) gets the rate-limit warning; session.onClose(fn(code, reason)) gets the close code and reason (4000 refused, 4001 kicked by Player::kick, 4290 for sending too much, and 4291 for a connection that would not read what the server sent).

Counters and objects to ask

AskWhat it tells you
game.stats() (server::Game::Stats)since the Game started, over every player including those who left: dropped (rate, ringFull, perType, total()), closedForRate, closedStalled, badRadius
player.dropped(), player.link(), player.crowdDelay(), focusDelay()per player: dropped messages; the measured link; how long the longest-waiting cell or focused entity waited
schedule.stats()per system: last and longest run in nanoseconds, runs (schedule)
ecs::dump(registry, entity)an entity as JSON (dump)
inputs.applied(c), queued(c), dropped(c), failed()the input queue of a client
interest.subscribed(c), deferred(c), budget(c), layoutsPending(c), entriesLast(c)what the last send did for a client
MemoryStore::saves(), erases(), size(topic)what a test store was asked to write
persistence.stats() (PersistenceStats)dirty and in-flight rows, batches sent, acknowledged, failed, timed out, refused
prediction.stats(), resetStats(), correction().serialclient side: prediction counters (cumulative: reset them before the part of a test you measure); a rising serial means the server disagreed
session.malformed(), sendsFailed(), layouts(), pollCount()client side: messages the replica refused, messages the connection refused, layouts applied, polls so far
GET /health on the WebSocket portanswers ok (Options::webSocket.healthPath, default /health; empty: 404)

Compile-time diagnostics

Most mistakes in a struct, a handler or a prediction are caught while compiling, with a message that says what to do. The repository checks the wording: each file in tests/compile_fail/ is compiled with -fsyntax-only as a ctest test (compile_fail_*) and must fail with the text its header comment quotes. The kinds you will meet: a struct declared inside a function (declared inside a function ... Declare the struct at namespace scope), a member a row cannot hold (has a type that a row cannot hold. Allowed: ...), a base class, bit-field or C array member (use std::array<T, N>), a List of Lists, more members than aether::reflect::kMaxMembers, an enum without a fixed underlying type, a struct that is not flat used as a component (is not a flat struct, so it cannot be a component), local component rules (must be movable, move constructor must be noexcept), a handler that returns a value (send the result as a message: send(player, Result{...})), an act over 255 bytes (THIS ACTION IS TOO BIG), a predicted step with a foreign row (every Row& parameter of fn must be one of the rows this prediction was made from), and a persisted struct with a List or an Entity (... convert<Saved>).

Building the SDK

Aether is a set of static C++23 libraries, built with CMake (3.24 or newer) and its dependencies fetched by Conan 2. build.sh does both:

./build.sh                       # Release build into build/, 4 parallel jobs
BUILD_TYPE=Debug JOBS=2 ./build.sh
RUN_TESTS=1 ./build.sh           # also runs ctest at the end

That script is three commands (it uses Ninja):

conan install . --build=missing -s build_type=Release -of build
cmake -S . -B build -G Ninja \
  -DCMAKE_TOOLCHAIN_FILE=build/build/Release/generators/conan_toolchain.cmake \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build -j4          # a bare -j can use all the memory of a small machine
Variable of build.shDefaultMeaning
BUILD_TYPEReleaseConan's and CMake's build type; the generated toolchain file is under build/build/<BUILD_TYPE>/generators
JOBS4parallel build jobs (-j)
RUN_TESTS01 runs ctest --test-dir build --output-on-failure at the end
Tip. Use -j4 (or the JOBS of build.sh) on a small machine: a bare -j uses all the memory, and several test executables compile large template headers.

Running tests

ctest --test-dir build -j4                       # everything
ctest --test-dir build -R Meadow                 # by name: Suite.TestName, a game's tests, compile_fail_...
ctest --test-dir build -R compile_fail -j4
ctest --test-dir build -N                        # list the tests, run none
ctest --test-dir build --output-on-failure

cmake --build build --target walker_test -j2     # build one game's test only
build/walker_test --gtest_filter='EasyLayer.*'   # or run a test executable yourself, filtered

Every gtest test is registered with ctest under its own name (gtest_discover_tests), as are the compile_fail_* checks, shared_header_stands_alone (<aether/shared.h> compiles alone and pulls in neither the registry nor any server header) and aether_game_generates_its_schema_header. The test executables are:

ExecutableWhat it holds
aether_teststhe SDK's tests (ring buffers, the ECS, Interest, the client, the older example games walker, driving, arpg, duel, portals, networking conditions ...)
guide_recipes_testthe recipes of the guide, compiled and run; includes only <aether/testing.h> on purpose
<game>_testone executable per example game with its own test: tag, chests, arena, farm, raid, kart (and walker, driving, arpg, duel, portals, built only on request)
aether_path_rate_teststhe estimate of a GNS connection's path rate against a model: pure logic
aether_gns_tests, aether_gns_client_teststhe GNS server, and a GNS client over a real server, a UDP relay, congestion and net conditions; their own executables because GNS keeps process-wide state. Only with AETHER_WITH_GNS.
aether_rune_teststhe rune adapter and games on a RuneStore. Integration tests start a broker when AETHER_RUNE_BROKER names its binary (a rune-auth test also needs AETHER_RUNE_CLI) and skip without it. Only with AETHER_WITH_RUNE.

The long variants of the network tests (more players, longer) are behind AETHER_LONG=1: AETHER_LONG=1 aether_tests --gtest_filter='NetConditions*'. To compile one test file alone for quick tries, configure with -DAETHER_DEV_TEST=tests/FooTest.cpp and build the target dev_test (it is EXCLUDE_FROM_ALL).

CMake options

OptionDefaultWhat it does
AETHER_BUILD_TESTSONbuilds the tests (needs GTest)
AETHER_BUILD_EXAMPLESONbuilds the example servers, clients and games, and defines aether_game
AETHER_BUILD_BENCHMARKSOFFbuilds the benchmarks
AETHER_BENCH_JOLTOFFalso builds crowd_bench and fight_bench (needs the Jolt physics package; see the comment at the benchmarks in CMakeLists.txt)
AETHER_WITH_GNSONbuilds the GameNetworkingSockets flavour (aether_gns, aether_client_gns)
AETHER_WITH_RUNEONbuilds the rune persistence adapter (aether_rune)
AETHER_DEV_TEST (cache string)emptya test file to build alone as dev_test
AETHER_SCHEMA_EXECUTABLE (cache)unsetthe aether-schema program, when it is not part of the build

The libraries

All are static. Link only what a program needs: a game client links aether_client alone and pulls in no server code and no uWebSockets.

LibraryHoldsLinks publiclyInclude / when
aether_clientecs::Type, the Replica (decoder of the tick message), the WebSocket client Connection, Session. No uWebSockets, no server code. A bot or game client links this alone.(include dir)<aether/client.h>
aetherthe server and the ECS: App, WorldProcessor, Registry, View, Interest, Layout, Persistence, Store, Schedule, Dump, the session pipelineaether_client (because ecs::Type lives there)<aether/server.h>, <aether/ecs.h>, <aether/testing.h>; uWebSockets is private to it
aether_gnsthe server over GameNetworkingSockets (server::GameSocket)aether<aether/server/GameSocketServer.h>; only with AETHER_WITH_GNS
aether_client_gnsthe client side of gns://aether_client<aether/client/GameSocket.h>; only with AETHER_WITH_GNS
aether_runethe rune persistence adapter, RuneStoreaether, rune::client<aether/ecs/RuneStore.h>; only with AETHER_WITH_RUNE. A game that does not persist links no rune client and no aeron.
aether-schema (executable), aether_schema_libthe contract file toolsee the contract file

The Conan recipe lists the libraries in this order, last one last, because static link order matters: aether, aether_gns (if GNS is on), aether_rune (if rune is on), then aether_client. aether uses ecs::Type, which lives in aether_client, so aether_client goes last.

Dependencies and the Conan recipe

RequirementVersionUsed by
uwebsockets20.71.0the WebSocket server (usockets with with_ssl=False: plain WebSocket; terminate TLS in front of the server)
gamenetworkingsockets1.4.1option with_gns (default True)
rune-client-cpp0.1.7option with_rune (default True)
entt3.15.0option with_benchmarks (default False); only ecs_iteration_benchmark
gtest1.16.0test requirement

The recipe is a static library named aether, version 0.1.0, with the options with_gns, with_rune and with_benchmarks. Its build() configures CMake with tests and examples off and GNS and rune following the options, and package() runs CMake.install(). The install puts aether and aether_client (and the GNS and rune libraries) in lib, the whole include/ in include, aether-schema in bin and AetherSchema.cmake in lib/cmake/aether.

No find_package(aether). The SDK's CMake installs the libraries and headers but no aether-config.cmake and no exported targets, so find_package(aether) is not provided by it. A project outside the repository takes the headers from include/, links the libraries in the order above (with the dependencies of the table), and builds with C++23.

aether_game: a game in the repository

The SDK's own games live in examples/<name>/. One CMake function, defined when AETHER_BUILD_EXAMPLES is on, wires a game up from a naming convention:

aether_game(<name> [TEST_IN_AETHER_TESTS])
ParameterMeaning
namethe game's name, lower case. Looks for examples/<name>/server.cpp, examples/<name>/client.cpp, an optional examples/<name>/<name>.aether and an optional test tests/<Name>Test.cpp (<Name>: the name with its first letter in capitals: chests gives tests/ChestsTest.cpp).
TEST_IN_AETHER_TESTSflag: the test is also listed in aether_tests (the older games). <name>_test is then built only on request (EXCLUDE_FROM_ALL) and not registered with ctest, so nothing runs twice.

It creates:

  • <name>_server, linked to aether, with examples/ as an include directory (so a game's own headers are included as "mygame/Shared.h");
  • <name>_client, linked to aether_client only;
  • <name>_test, its own small gtest executable (with tests/ and examples/ as include directories, linked to aether and GTest::gtest_main), registered with ctest through gtest_discover_tests;
  • if examples/<name>/<name>.aether exists: <name>_messages, the generated <name>/Messages.h (see aether_schema below), linked into the server, the client and the test. A <name>_messages target you made yourself before calling aether_game is used as it is.

The files are found with CONFIGURE_DEPENDS, so a file you create after the first configure is picked up by the next cmake --build, with no re-run of cmake. A missing server.cpp or client.cpp is a configure warning: aether_game(<name>): examples/<name>/server.cpp and examples/<name>/client.cpp must both exist; until they do there is no <name>_server, <name>_client or <name>_test target. Create them, then build again (no need to re-run cmake). and no targets; a missing test is only a status message. To add a game, create the files and add one line to the AETHER_BUILD_EXAMPLES block of CMakeLists.txt.

Name the gtest suite like the game (TEST(Mygame, ...)) so ctest -R Mygame works, and build one game and its test with cmake --build build --target mygame_test -j2; build/mygame_test --gtest_filter=Mygame.Persist*. The games registered in the SDK are walker, driving, arpg, duel and portals (with TEST_IN_AETHER_TESTS), and tag, chests, arena, farm, raid and kart (each with its own test). The plain example targets are echo_server, assemble_server, view_server and view_client.

aether_schema: the contract header

include(lib/cmake/aether/AetherSchema.cmake)     # installed by the SDK; inside the SDK's own build it is already included
aether_schema(pond_messages SCHEMA schemas/pond.aether JS ${CMAKE_BINARY_DIR}/web/gen)
target_link_libraries(my_server PRIVATE pond_messages aether)
ArgumentMeaning
targetthe name of the INTERFACE library to create; link it to everything that includes the header (#include <pond/Messages.h> for pond.aether).
SCHEMA <file.aether>required; relative to the calling CMakeLists.txt's directory. The header is <stem>/Messages.h, in the include directory <binary dir>/aether_schema/<target>/include.
JS <dir>optional: also writes <dir>/<stem>.js, .d.ts and .schema.json (a relative dir is under the binary directory).
MAX_MESSAGE_BYTES <n>optional: passed to the generator as --max-message-bytes.

The generator runs at build time whenever the schema changes, through a custom target <target>_generate that is part of ALL (so the JavaScript is produced even if no C++ target links the header). A schema with an error fails the build with the generator's file:line:col: error: ... lines. It leaves a file alone when its content would not change, so nothing that includes it recompiles. The generator is the target aether-schema when it is part of this build, else the program in the cache variable AETHER_SCHEMA_EXECUTABLE, else one found on the PATH. Errors: aether_schema(<t>): SCHEMA <file.aether> is required, ... unknown arguments: ..., ... the schema <path> does not exist.

The tool, its command line and the file grammar are on the data page.

Reference

Everything in <aether/testing.h> (namespace aether::test), the store counters tests use, and every CMake function, option and script. The header pulls in aether/client.h, aether/client/Session.h, aether/server/App.h, Game.h and Host.h: a game test needs nothing else.

test::Config

struct Config {
    int tickRate = 50;
    int flushRate = 200;
    server::RateLimit rate = {};
    int stalledSeconds = 300;
    int linkReportIntervalMs = 250;
    const server::Transport* transport = nullptr;
    server::GameSocketOptions gameSocket = {};
};
FieldType, defaultValid values and effect
tickRateint = 50ticks a second the world runs at (Options::tickRate). A Host's games tick at this rate divided by a whole number. An invalid Options value throws std::invalid_argument("aether::server::Options::<field>: ...") from the constructor.
flushRateint = 200flush requests a second of the network thread (Options::flushRate, clamped to at least 1); the default of Options is 30.
rateserver::RateLimit = defaultsfields perSecond (30; 0 = no limit at all), burst (60; 0 = perSecond), bytesPerSecond (16384; 0 = no byte limit), burstBytes (32768; 0 = bytesPerSecond), violationWindowMs (2000), perTick (16; 0 = read all). First violation: the message is dropped and the client's onWarning is called; a second inside the window is dropped silently; a third closes the connection (4290).
stalledSecondsint = 300a connection whose output has been blocked this long is closed (close code 4291); 0 = never; negative is invalid.
linkReportIntervalMsint = 250how often the world gets each client's byte rates and backlog; 0 = never (the Link rates stay 0); negative is invalid.
transportconst server::Transport* = nullnull: WebSocket on a free port. Else the transport to listen on, e.g. a server::GameSocket (GameSocketServer.h, link aether_gns); it only has to live through the constructor. url() then gives a gns:// address. Only one GameSocket server may run at a time in a process.
gameSocketserver::GameSocketOptions = {}Options::gameSocket (fake loss and lag, ...); read only with a transport that is a GameSocket.

The other Options keep their defaults; handleSignals is false.

test::Server<World>

template <class World> class Server {
public:
    World world;
    Server();
    template <class... Args> explicit Server(Args&&... args);              // World's arguments; the first is not a Config
    template <class... Args> explicit Server(Config config, Args&&... args);
    ~Server();
    Server(const Server&) = delete;   Server& operator=(const Server&) = delete;
    std::string url(std::string query = {}) const;
    std::uint16_t port() const;
    void stop();
    template <class F> auto run(F fn);
};
ItemMeaning
Worlda server::Game or a server::Host: anything the App serves that also has post(fn) (run() uses it).
constructorsbuild world from args, then start an App listening on port 0 (a free one) on a thread of its own, and wait for it. Throws std::runtime_error: test::Server: the server did not start listening in 10 s; test::Server: the world's onStart threw or no port could be bound (see stderr).
worldthe World itself. Touch it only before the first client connects, or through run().
url(query)ws:// (or gns:// with a transport) address of the server on the loopback interface and its port, with / and, if query is not empty, ?query. A leading ? in query is removed: url("name=Ayla") and url("?name=Ayla") are the same. A Host game: url("game=duel"). Several parameters: "name=Ann&game=arena".
port()the bound port (never 0 after the constructor returns).
stop()stops the app and joins its thread; harmless if already stopped. The destructor calls it. A FileStore or MemoryStore the game saves into has what the shutdown saved once it returns.
run(fn)runs fn(World&) on the world thread between two ticks and returns its result (the return type is decltype(fn(world)); void works). Throws std::runtime_error if it does not run in 10 s: test::Server::run: the world thread did not run it in 10 s (is the server stopped?). An exception thrown by fn is rethrown to the caller.

pump, pumpFor, waitFor

template <class... Args> bool pump(Args&&... args);        // pump(session..., until [, timeout])
template <class Rep, class Period, class... Args>
void pumpFor(std::chrono::duration<Rep, Period> duration, Args&&... args);   // pumpFor(duration, session..., [each])
template <class Pred> bool waitFor(Pred pred, std::chrono::milliseconds timeout = std::chrono::milliseconds(10000));
template <class Pred> bool waitFor(Pred pred, int timeoutMs);
CallArgumentsBehaviour
pumpone or more sessions (client::Session, or anything with poll(int)); then until, a callable returning bool; then optionally a std::chrono::duration timeout (default 10 s)loops: if until() is true return true; else poll(1) every session (1 ms each). When the time is up, returns until() once more. Compile errors (static_assert): fewer than two arguments (pump(session..., until [, timeout]): at least one session and the function), a timeout without a function or sessions, a function that does not return bool.
pumpFora duration; one or more sessions; optionally a last argument, a callable with no parameters, called after each round of pollspolls every session (poll(1)) until the duration has passed; returns nothing. Compile error with no session: pumpFor(duration, session..., [each]). Note: the "each" argument is recognised by being callable with no arguments, so a session type must not be.
waitForpred, a callable returning something convertible to bool; timeout in std::chrono::milliseconds (default 10000) or an int of millisecondschecks pred() every 2 ms, sleeping in between, polling nothing; returns true as soon as it is, else pred() once more after the timeout.
ASSERT_TRUE(test::pump(a, b, [&] { return a.ready() && b.ready(); }));
test::pumpFor(250ms, a, [&] { me.input(Move{1, 0}); });
ASSERT_TRUE(test::waitFor([&] { return store.saves() == 1; }, 2000));

The stores a test uses

class MemoryStore : public Store {
    std::size_t size(std::string_view topic) const;
    std::size_t saves() const;
    std::size_t erases() const;
};
class FileStore : public MemoryStore {
    explicit FileStore(std::filesystem::path path, bool sync = false);
    const std::filesystem::path& path() const;
};
ItemMeaning
MemoryStorea store in this process: for tests, local runs and "a second World in the same process". The rows live as long as the object.
size(topic)how many keys a topic holds.
saves(), erases()what the game has written since the store was made: rows put (a new row or a changed one), rows removed. Reading does not count; neither does a FileStore's reading of its file when it opens.
FileStore(path, sync = false)a MemoryStore that also appends every change to path and reads the file back when it is made: the rows survive the process. The file is a log (a record per change), rewritten smaller when it opens and most of it is old values; a cut tail (a crash in the middle of a write) is ignored. Throws std::runtime_error for a file that cannot be opened or is not a store file, and from write() when the disk refuses (the Game reports it on stderr and tries again at the next tick). sync = true also asks the OS to put each batch on the disk (fsync): slower, and what a power cut needs; a crash of the process needs only the default.

CMake reference

Functions

aether_game(<name> [TEST_IN_AETHER_TESTS])
aether_schema(<target> SCHEMA <file.aether> [JS <dir>] [MAX_MESSAGE_BYTES <n>])

aether_game is defined in CMakeLists.txt inside the AETHER_BUILD_EXAMPLES block; aether_schema is defined by AetherSchema.cmake, which the SDK's build includes and installs to lib/cmake/aether. Parameters, outputs and errors: aether_game and aether_schema above.

Targets

TargetKindCondition
aether_client, aetherstatic librariesalways
aether_gns, aether_client_gnsstatic librariesAETHER_WITH_GNS
aether_runestatic libraryAETHER_WITH_RUNE
aether-schema, aether_schema_libexecutable, static libraryalways (add_subdirectory(tools/schema))
echo_server, assemble_server, view_server, view_clientexecutablesAETHER_BUILD_EXAMPLES; view_* get GNS support with AETHER_WITH_GNS (./view_server --gns 27015, ./view_client gns://<host>:27015/)
<game>_server, <game>_client, <game>_test, <game>_messagesfrom aether_gameAETHER_BUILD_EXAMPLES (_test also needs AETHER_BUILD_TESTS)
aether_tests, guide_recipes_test, aether_path_rate_tests, aether_gns_tests, aether_gns_client_tests, aether_rune_tests, dev_testtest executablesAETHER_BUILD_TESTS (and the GNS / rune options; dev_test needs AETHER_DEV_TEST)
link_measure, gns_probeexecutablesAETHER_BUILD_TESTS and AETHER_WITH_GNS (the CMake file builds them there)
benchmarksexecutablesabove

Scripts

ScriptArguments
build.shenvironment BUILD_TYPE (Release), JOBS (4), RUN_TESTS (0)
tests/netns/link.sh[--rate B/s] [--uplink-rate B/s] [--delay ms] [--jitter ms] [--loss pct] [--loss-up pct] [--queue bytes] [--reorder pct] [--aqm] [--at TIME key=value ...] -- command...; environment AETHER_LINK_VERBOSE=1; exit 77 if the namespaces cannot be made
tests/netns/baseline.sh<path to link_measure> [seeds=5] [condition,...]
tests/netns/scenarios.sh<path to link_measure> [seeds=5] [cell,cell,...]; exit 1 if a cell failed, 77 if the link cannot be made
benchmark/run_{budget,crowd,fight,sendperf}.sh<build dir> <results dir> [extra args]

Environment variables the tests and tools read

VariableRead byEffect
AETHER_LONGthe network condition teststhe long variants (more players, longer)
AETHER_RUNE_BROKERthe rune integration tests, ecs_rune_benchmarkpath of a rune broker binary to start; without it those tests skip
AETHER_RUNE_CLIthe rune auth testspath of the rune command-line client
AETHER_LINK_VERBOSElink.shprint tc qdisc show
AETHER_NETNS_SERVER_ADDR, AETHER_NETNS_PEERset by link.sh for its commandwhere to bind the server; the namespace to setns into for the client socket