E‑Jam
A test bench for network switches, spread across many machines.
Fig. 01 · Traffic on a line, one burst per beat. The shape of the bench it stands for is drawn properly in Fig. 05. Click or tap to send a burst. On the beat, it passes the switch in amber. Off the beat, it turns red and stops there.
01 · The problem
Proving a switch is sound took an afternoon and a person
A network switch is the box every packet in an office passes through, and a new one has to prove it can carry full traffic without dropping or delaying anything. The proof is a standard set of measurements (throughput, latency, frame loss) at several frame sizes, each repeated many times. Done by hand, one full pass took an engineer more than two hours, and the hand was the slow part.
Two RFCs define what those measurements mean. RFC 2544 finds throughput by binary search on the send rate: every trial runs at least 30 seconds and the final one at least 60, and a trial passes only if nothing is dropped. Latency comes from a single tagged frame dropped into a 60-second stream, timestamped on the way in and the way out, over at least 20 trials. Frame loss is at least 10 trials per frame size, starting at the full rate and stepping down. RFC 2889 covers the switching cases on top of that, and RFC 1242 and RFC 2285 supply the vocabulary both of them use.
None of that is difficult. All of it is long, repetitive and exactly the kind of work a person does slightly differently the fourth time.
02 · What we built
One control plane, as many test nodes as you have
You describe a test once. Every machine in the bench runs its share of it, and every process reports on the same bus, so the client can draw the result while the run is still going.
REST
starts and stops a run on every node
launches the processes in Docker
packets to the switch and back
Avro-typed topics, checked against the schema registry
The part I worked onThe rest of the team
What the user controls
One frame size or a mix of them in a chosen ratio, from 64 bytes up to 1518. Traffic that runs back to back or in bursts, with the inter-frame gap under your hand. A share of broadcast frames, to see how the switch copes with flooding. A PRNG seed and a payload type, so a run can be repeated exactly. Four presets cover the usual jobs: throughput, latency, frame loss rate and packet integrity.
How a packet proves itself
Every frame carries a two-byte stream id, an eight-byte sequence number, a timestamp the switch writes as the frame passes, and a four-byte CRC. That is the whole trick: with an id and a sequence number the verifier knows which stream a frame belongs to and whether any went missing, arrived twice or arrived out of order, and with the switch's own timestamp it can price the delay the switch added rather than the delay the network added.
RFC 2544 throughput: the fastest send rate at which nothing is lost.
Fig. 05 · E‑Jam with the traffic on. Generator nodes on the left fire packets down their own stream to the switch under test, and the switch forwards each one to the verifier that is expecting it. Every packet turns amber for the instant it is inside the switch. That amber is the stamp the verifier prices the delay from. Not every frame comes out the far side, and the tally is what notices, which is the whole reason a frame carries a sequence number. The layout and the counters describe this drawing, not a measured run. The nodes are buttons. Turn a generator off and its streams stop: nothing more is sent, so nothing on those lanes can be lost. Turn a verifier off and the stream still crosses the switch, but nobody checks the far end, so every frame on it counts as lost. Add or remove generators and verifiers with the buttons under the figure. There is always a stream for every node: one generator sends to every verifier, and one verifier checks every generator. Click anywhere else on the figure to send a burst. Find throughput runs the RFC 2544 throughput test on this drawing: a binary search on the send rate, starting at line rate, for the fastest rate that loses nothing, then one longer final trial at that rate. The switch gets a hidden limit for every run and says what it was at the end. During the test the drawing's random loss is off, so only that limit drops frames, and the nodes are locked while it runs.
03 · My part
The Admin Client: the control plane, the stream, and the client
Five of us built E‑Jam. The three I worked on sit together in the repository under one directory called Admin Client, and they are the route a number takes: a process on a test node counts something, and a chart on somebody else's desk has to show it.
Center Point is the Rust service on Actix Web that everything else talks to. Its REST API is the only way a test is created, started, stopped or inspected, and behind that API every stream, device and process moves through an explicit state machine rather than a pile of booleans. A stream is created, sent to the nodes, queued, started, and then finishes, stops or errors. Each of those edges is a named transition that can be refused. Being able to say "this stream cannot go from Queued to Finished" is what makes an unattended run trustworthy: a test that ends in a state nobody expected is a test whose result you cannot use.
Center Point also consumes the statistics topics, so it is the one place that knows what every node is seeing, and the one place the client has to ask.
The Kafka layer is how anything that happens on a node reaches the screen at all. Each node's System API produces its statistics onto topics. The messages are Avro, registered against a schema registry, so a producer and a consumer that disagree about the shape of a statistic fail at the registry instead of quietly halfway through a run. The stack itself comes up from one compose file (Zookeeper, the brokers, the registry, ksqlDB), so a new bench has a streaming layer before it has a test.
The Flutter client is the end of that path: it configures a test and charts the results as they arrive. The team moved it off Electron mid-project, which the project documentation records as the riskiest call we made. It was also the right one. The UI got fast enough to draw live data without falling behind it.
The stack, end to end
Five components, four languages, and which of them I worked on. Nobody wrote all of it.
- Center Point my part
- Rust · Actix Web. The control plane, and the only thing the client ever talks to. It is also the one process that has seen what every node has seen.
- Streaming my part
- Apache Kafka · Avro · schema registry. The bus every node's statistics travel on, typed at both ends. Zookeeper, the brokers, the registry and ksqlDB all come up from one compose file.
- Admin client my part
- Dart · Flutter. The cross-platform desktop client: configure a test, watch it draw. It replaced an Electron prototype halfway through the project.
- System API
- Java · Spring Boot. One per test node. It takes what Center Point sends, starts and stops the processes in Docker on that machine, and produces their statistics onto Kafka.
- Generators, verifiers, gateway
- C++. The processes that put frames on the wire and check what comes back, one container each, handing their statistics up through named pipes.
- Switch under test
- SNMP. Not ours: the device being measured, watched passively between runs.
GitHub's language breakdown puts about a tenth of the repository under C. There is no C component: the repository holds no hand-written C source, and that tenth is header and build files counted that way.
04 · What changed
The measurements
Each figure below names where it comes from. The project documentation records no packets-per-second, node-count or Kafka throughput numbers, so this page does not carry any.
- One full test pass, from an afternoon to a coffee break.
- 2 h+down to~10minCV · E‑Jam
- Tests that were run by hand now run themselves and grade the result.
- 5+casesCV · E‑Jam
- Commits, the largest share in a team of five developers.
- 256/ 548GitHub · contributors API
- Dart · the admin client
- 31%
- C++ · generators and verifiers
- 20%
- Java · the per-node system API
- 16%
- Rust · Center Point
- 12%
05 · The team
Five developers, two-week sprints
A five-developer graduation project, run in two-week sprints with a product owner and a scrum master alongside us, supervised at Cairo University by Dr Mohammad El-Ramly. I worked on the control plane, the streaming layer and the client. The repository is public under MIT, and the documentation, the diagrams and the screenshots on this page all come out of it.
Source on GitHub Watch the walkthrough Back to the rest of the work