Server stalls from a growing history: before and after
Obeya's server runs on one thread. Several of the reads it repeats on every change, every Koordinator command and every pass over the shared pages went through whole tables, so they took longer with every day of use. While one ran, every page and the phone froze.
What each round costs over a history ten times today's
A synthetic database with ten times the history of the owner's Obeya: 1.6 million events over 400 days, 9,000 archived cards and 4,500 shared pages on one canvas, and 10,000 Koordinator exchanges. The unit is one rebuild of the canvas view on a canvas with the same cards and no history (about 1.5 ms on this laptop), measured right beside each round, so the budgets hold on slower machines too. The scale is logarithmic.
Table view
On a copy of today's database (55 MB)
The same reads on a copy of the real database, warm cache. A busy machine that has to read pages back from disk or swap stretches these into the stalls the log shows.
| Read | Before | After |
|---|---|---|
| Koordinator brief: what happened in the last 14 days | 28 ms | 0.14 ms |
| Learner: what the owner said in the last days | 24 ms | 0.10 ms |
| Steps the Koordinator waits to take (every change) | 0.06 ms | 0.002 ms |
Where the 99 stalls went
| Cause (top of the sampled stack) | Stalls | Seconds | Now |
|---|---|---|---|
| Synchronous process calls (git, ssh) | 20 | 17.1 | Most fixed before this card; the rest moved off the thread, a test counts what remains |
| Synchronous file calls (demo site, demo sweep, stats) | 19 | 13.3 | Moved to async reads; what stays is listed with a reason |
| Notice of outdated pages (every shared-card change) | 14 | 19.2 | Shared cards kept up to date from a log of their writes |
| Rebuilding the canvas view | 9 | 6.0 | Project costs and workstreams read by index |
| Steps the Koordinator waits on | 9 | 7.9 | Partial index |
| Checking the sites' shared pages | 9 | 11.2 | No archived cards built; one target per site |
| Koordinator brief (history of the last days) | 6 | 5.1 | Read by time through an index |
| Learner (the owner's recent words) | 5 | 4.4 | Read by time through an index |
| Other | 8 | 7.8 | Not addressed |
Guard against new synchronous calls
A test counts every synchronous file and process call (every function ending in Sync) in the server's code against a list: per file, how many of each it may make and why. A new call fails the test until it is made asynchronous or listed with a reason; a call that goes must leave the list, so the list only shrinks. 48 files are listed today, most with startup, script or single-stat reasons.
Moved off the server thread in this change: a canvas's repository settings (no fallback read with git), the first canvas's folder, a demo site page's captions, posters and sound, the demo sweep's settings file, and a repository's theme for a page about to go out.