Director’s commentary · The Ham Sandwich

Holding 60 fps on a phone

Quality tiers from a GPU estimate, a governor that steps down when frames run slow, a render loop that idles at 30 fps and stops when hidden, budgets that fail the build, and fog doing the work of a whole street.

The target in the spec is 60 fps on a mid-range phone at the medium quality tier, and never below 30 on the low one. A phone is a small, hot computer running on a battery, and a point-and-click adventure spends most of its life standing still, waiting for you to read a line or make up your mind. Both of those facts shaped the engine more than any shader did. Most of the work of holding a frame rate turned out to be deciding what not to draw.

Start in the right place, then correct

Before the first frame, the game makes a guess. detect-gpu looks up the device's graphics chip against benchmark data, which the site serves itself from /benchmarks rather than fetching from a third party, and gives it a rough tier. That, plus whether WebGPU is actually available, picks where the game starts. A chip it doesn't recognise is usually a new one, so that starts at medium rather than low. If the whole thing takes more than five seconds, the game gives up guessing and starts at medium, or low without WebGPU.

const SETTINGS: Record<QualityTier, TierSettings> = {
  high: { dpr: 2, post: "full", shadows: true, extras: 1, textureMax: 1024 },
  medium: { dpr: 1.5, post: "light", shadows: true, extras: 0.75, textureMax: 1024 },
  low: { dpr: 1, post: "minimal", shadows: false, extras: 0.4, textureMax: 512 },
  plain: { dpr: 1, post: "minimal", shadows: false, extras: 0, textureMax: 512 },
};
// …
export function tierFromGpu(gpu: GpuEstimate, webgpu: boolean): QualityTier {
  if (gpu.type === "WEBGL_UNSUPPORTED") return webgpu ? "low" : "plain";
  // An unlisted GPU is usually a new one, and the governor can only step down from the start.
  if (gpu.type === "FALLBACK") return "medium";
  if (gpu.tier >= 3 && webgpu) return "high";
  return gpu.tier >= 2 ? "medium" : "low";
}

Each tier caps the device pixel ratio, which is the single biggest lever on a phone: a pixel ratio of 2 draws four times as many pixels as a ratio of 1. It also decides the post-processing. Medium drops depth of field and runs bloom at quarter resolution. Low keeps only the colour grade, the vignette and the fade, so every chapter still has its colour, just not its extras.

A guess is only a guess, so the frame governor watches real frame times. It keeps a smoothed average and steps down a tier after about two seconds slower than 50 fps, and back up after about eight seconds faster than 58. The gap between those two numbers stops it flapping between tiers, and it never climbs above the tier it started at, so a device that was guessed at medium never talks itself into high. Very long frames, like the one after a tab switch, are ignored rather than counted as load.

const STEPS: QualityTier[] = ["high", "medium", "low"];
const EMA_ALPHA = 0.05;
const SLOW = 1 / 50;
const FAST = 1 / 58;
const DOWN_AFTER = 2;
const UP_AFTER = 8;
/** Longer frames are tab switches or hitches, not steady load. */
const MAX_DT = 0.25;
// …
sample(dtSeconds: number): void {
  if (this.index < 0 || !(dtSeconds > 0) || dtSeconds > MAX_DT) return;
  this.ema = this.ema === 0 ? dtSeconds : this.ema + (dtSeconds - this.ema) * EMA_ALPHA;
  this.slowFor = this.ema > SLOW ? this.slowFor + dtSeconds : 0;
  this.fastFor = this.ema < FAST ? this.fastFor + dtSeconds : 0;
  if (this.slowFor >= DOWN_AFTER && this.index < STEPS.length - 1) this.step(1);
  else if (this.fastFor >= UP_AFTER && this.index > this.ceiling) this.step(-1);
}

Don't draw what isn't moving

The canvas renders on demand rather than in a constant loop. While anything is moving (for four seconds after your last input, during a minigame, while a chapter loads, or while someone's walking), it asks for a frame on every display refresh. When only idle animation is left, the breathing and blinking, it asks on every other one, which is 30 fps on most phones and halves the work while you're reading. When the tab is hidden, it stops asking at all. The audio scheduler stops with it.

// Frame pacing: 60 fps while anything moves, 30 fps when only idle animation
// is running (spec §7.8), paused while the tab is hidden.
const lastActivity = useRef(performance.now());
useEffect(() => {
  let raf = 0;
  let odd = false;
  const bump = () => (lastActivity.current = performance.now());
  const unsub = useUi.subscribe(bump);
  window.addEventListener("keydown", bump);
  const loop = () => {
    raf = requestAnimationFrame(loop);
    if (document.hidden) return;
    const s = ui.get();
    const active =
      performance.now() - lastActivity.current < 4000 ||
      !!s.minigame ||
      s.mode === "loading" ||
      [...game.world.actors.values()].some((a) => a.mover.moving);
    odd = !odd;
    if (active || odd) invalidate();
  };

One I caught while writing this page: the governor's own documentation says to feed it only frames rendered at full rate, but the Stage was sampling every frame it rendered, including the deliberate 30 fps idle ones. Sit still long enough and those looked like a slow device, so the tier could step down while you were just reading. The pacing loop now records whether it's running at full rate, and only those frames reach the governor.

useFrame((_, delta) => {
  const dt = Math.min(delta, 0.1);
  game.frame(dt);
  // Depth of field always focuses on what the shot is looking at.
  const director = game.world.director;
  if (director && pipelineRef.current) pipelineRef.current.set({ dofFocus: director.focusDistance() });
  pipelineRef.current?.render();
  if (fullRate.current && useSave.getState().settings.quality === "auto") governorRef.current?.sample(dt);

Budgets that fail the build

Performance you only check by hand is performance that slowly gets worse, so the budgets from the spec are a script, and CI runs it after every build. Every prop has to stay within 10 percent of its triangle budget. Every chapter's props have to download in under 6 MB, and the prologue's in under 4. The JavaScript for the game route has to be under 500 KB gzipped, three.js included, and the plain pages have to be under 150 KB with no WebGL renderer in them at all.

export const LIMITS = { trisTolerance: 1.1, chapter: 6 * MB, prologue: 4 * MB, gameJs: 500 * KB, plainJs: 150 * KB };
// …
export function checkChapters(manifest: PropManifest, specs: PropSpec[]): Check[] {
  const { bytes, unknown } = chapterBytes(manifest, specs);
  const checks: Check[] = CHAPTERS.map((c) => {
    const limit = c === "prologue-ham-sandwich" ? LIMITS.prologue : LIMITS.chapter;
    return { name: `download: ${c}`, value: fmt(bytes[c]), limit: fmt(limit), ok: bytes[c] < limit };
  });
  if (unknown.length) checks.push({ name: `unknown chapters: ${unknown.join(", ")}`, value: String(unknown.length), limit: "0", ok: false });
  return checks;
}

Right now there's plenty of room. The prologue's props come to 878 KB against its 4 MB budget, and the biggest chapter, Now, is 1.18 MB. How the props got that small is its own page, 59 props from Meshy.

Fog is cheaper than a street

The prologue is a three-year-old on a sunny doorstep, and what a three-year-old notices is the sandwich, the step and the bird. Everything past the gate fades into sunlight. That's the memory, and it's also the cheapest possible way to draw a street. Fog starts seven metres out and is total by twenty-six, in the same warm white as the sky, and depth of field, refocused every frame on whatever the shot is looking at, softens everything else. So the street is a few simple boxes and a lamppost, and the fog does the rest.

const PROLOGUE = look("prologue", {
  // Summer sun: blown-out highlights, only what a three-year-old notices in focus, the street fading to white
  post: ["grade", "dof", "bloom", "vignette", "grain"],
  params: {
    exposure: 1.15, contrast: 0.92, saturation: 1, temperature: 0.3, tint: [1, 0.98, 0.93],
    bloomStrength: 0.55, bloomThreshold: 0.9, bloomRadius: 0.6,
    dofFocus: 1.4, dofAperture: 0.05, dofMaxBlur: 0.012,
    vignette: 0.22, grain: 0.16,
  },
  uiSkin: "crayon",
  background: "#fff3d6",
  fog: { color: "#fff1d2", near: 7, far: 26 },

What's next

Textures. They're WebP for now, because the tool for KTX2 isn't installed yet, and WebP is small to download but expands to full size in the phone's graphics memory. KTX2 stays compressed on the GPU, which is what a phone really wants, and switching the optimiser over is a small change once the tool is there.

And real devices. Each session sends one coarse frame-rate bucket after ten seconds of play, with no cookies and nothing that identifies anyone, and those buckets will show me where the tier guesses are wrong far better than any phone on my desk can.