Director’s commentary · The Ham Sandwich

59 props from Meshy

The pipeline behind every object in the game: one style preamble, a concept image, Meshy-7 image-to-3D, and an optimiser that turns 59 generated models into 4.7 MB a phone can download. Plus how the hero prop gets eaten.

Every object you can pick up, look at or walk past in the game started as a sentence. There are 59 of them so far, from a floppy disk to a server rack, and they all came through the same pipeline: one shared style preamble, a concept image, Meshy-7's image-to-3D with a target polycount, and a build script that turns whatever comes back into something a phone can download. All 59, optimised, come to 4.7 MB. The most important of them is a ham sandwich.

One preamble, 59 sentences

Generated assets drift. Ask for a floppy disk on Monday and a floppy drive on Tuesday and you get two objects from two different games. The fix is boring and it works: every prop is described with the same preamble, written into the style bible before anything was generated, with only the object itself swapped in. It fixes the proportions, the finish, the colours, the camera angle and the lighting, and it rules out logos and lettering, which keeps the game on the right side of the name-but-don't-show rule for real brands.

"preamble": "Stylised 3D game prop, {OBJECT}. Chunky toy-like proportions, softly rounded edges, hand-painted texture look, soft matte materials, gently muted warm colours, simple readable shapes. No logos, no brand names, no text or lettering. One single object, fully in frame, centred, three-quarter view from slightly above, plain light grey background, soft even studio light.",
…
{ "id": "ham-sandwich", "batch": 1, "chapters": ["prologue-ham-sandwich", "we-made-our-own", "now"], "tris": 5000, "hero": true, "size": { "longest": 0.14 }, "desc": "a ham sandwich on soft white bread with a thick slice of pink ham peeking out, cut diagonally into two triangles, the two halves lying side by side, a small child's lunch" },

Each entry also carries what the rest of the pipeline needs: which chapters use the prop, a triangle budget, whether it's a hero, and its real-world size in metres, as either a height or a longest side.

Concept first, then 3D

Each prop starts as a concept image from its description and the preamble, which is cheap to look at and cheap to throw away. Only a concept I'm happy with goes on to Meshy-7's image-to-3D, with a target polycount near the prop's budget and textures on. Together that's about 36 credits a prop, a little over two thousand for the lot.

The sandwich's concept came back better than I described it: two triangles of soft white bread with a golden crust, cut on the diagonal and laid side by side, a thick fold of pink ham in each, sitting on a square of pale, slightly stained paper. It also has a crisp edge of lettuce and a sliver of something yellow that the description never asked for. Generators like to garnish. For a three-year-old's lunch in 1987 that's a liberty, but it reads as a sandwich from across the doorstep, which is what matters, and it stayed.

The sandwich is the hero prop, because it opens the game, comes back in Disk 2 and closes the game in Now. It has a 5,000-triangle budget, the top of the range for props, and like the other six heroes it gets 1024-pixel textures. Everything else gets 512.

Making it web-sized

What comes out of Meshy is built for a desktop viewer, not a phone. The sandwich arrived as a 2.7 MB file. The optimiser, built on glTF Transform, takes it apart and puts it back together: merge duplicates, prune anything unused, weld the vertices, then simplify with meshoptimizer until it's under its triangle budget, relaxing the error bound a step at a time so it only gets as rough as it has to. Then it normalises the model: centred, sitting on the floor at y = 0, and scaled to its real-world size, because generated models come back at whatever scale they like. Last come WebP textures and Meshopt compression.

async function build(io: NodeIO, spec: PropSpec): Promise<{ entry: PropEntry; srcTris: number }> {
  const doc = await io.read(PATHS.src(spec.id));
  doc.setLogger(new Logger(Logger.Verbosity.WARN));
  const srcTris = countTris(doc);

  await doc.transform(dedup(), prune(), weld());
  let tris = countTris(doc);
  // Relax the error bound step by step until the budget is met.
  for (const error of [0.01, 0.05, 1]) {
    if (tris <= spec.tris) break;
    await doc.transform(simplify({ simplifier: MeshoptSimplifier, ratio: spec.tris / tris, error }));
    tris = countTris(doc);
  }

  const size = normalise(doc, spec);
  const max = spec.hero ? 1024 : 512;
  await doc.transform(
    textureCompress({ encoder: sharp, targetFormat: "webp", resize: [max, max] }),
    meshopt({ encoder: MeshoptEncoder, level: "high" }), // reorders, quantizes and applies EXT_meshopt_compression
  );

  const glb = await io.writeBinary(doc);
  writeFileSync(PATHS.out(spec.id), glb);
  const hash = createHash("sha1").update(glb).digest("hex").slice(0, 8);
  // The hash busts the immutable /assets cache when a prop is rebuilt.
  const entry: PropEntry = { url: `/assets/props/${spec.id}.glb?v=${hash}`, size, tris, bytes: glb.byteLength, hero: !!spec.hero };
  return { entry, srcTris };
}

The sandwich went from 5,086 triangles to 5,000 and from 2.7 MB to 184 KB. The URL in the manifest carries a hash of the file's contents, because the site tells browsers to cache props forever, and a rebuilt prop would otherwise never reach anyone who'd visited before.

Loading, placeholders and eating the hero

In the game, each prop is fetched once and cloned for every use, so clones share geometry, materials and textures. Loads are reference-counted, so a prop can be disposed as soon as nothing is holding it, although at 4.7 MB for the whole set the game doesn't bother yet: once a prop has loaded, it stays. If a prop hasn't been built yet, or fails to load, it becomes a soft rounded box of the right size, taken from props.json. That meant every chapter could be laid out and played long before its props existed.

export function releaseProps(ids: string[]): void {
  for (const id of new Set(ids)) {
    const entry = cache.get(id);
    if (!entry || --entry.refs > 0) continue;
    cache.delete(id);
    void entry.source.then((root) => dispose(root, entry.textures));
  }
}
// …
function placeholder(id: string, reason: string): THREE.Object3D {
  if (!warned.has(id)) {
    warned.add(id);
    console.warn(`[props] ${id}: ${reason}; using a placeholder box`);
  }
  const [x, y, z] = propSize(id) ?? [0.3, 0.3, 0.3];
  const geometry = new RoundedBoxGeometry(x, y, z, 3, Math.min(x, y, z) * 0.2);
  geometry.translate(0, y / 2, 0);
  const mesh = new THREE.Mesh(geometry, new THREE.MeshStandardMaterial({ color: "#b98d5f", roughness: 1 }));
  mesh.name = `${id}-placeholder`;
  const root = new THREE.Group();
  root.name = id;
  root.userData.placeholder = true;
  root.add(mesh);
  return root;
}

The sandwich is also the only prop in the game that gets eaten, and I didn't want four models for four bites. So the prologue gives its material a TSL mask. Each bite is three overlapping circles in the sandwich's own space; any fragment inside a circle is discarded, which leaves a scalloped row of teeth marks in the bread. The circles are uniforms with a radius of zero until that bite happens, so taking a bite is setting three numbers.

const BITES: { c: [number, number]; r: number }[][] = [
  [{ c: [0.07, 0.02], r: 0.026 }, { c: [0.064, -0.012], r: 0.024 }, { c: [0.076, 0.05], r: 0.022 }],
  [{ c: [-0.07, 0.03], r: 0.03 }, { c: [-0.066, -0.01], r: 0.028 }, { c: [-0.06, 0.06], r: 0.024 }],
  [{ c: [0.02, 0.07], r: 0.045 }, { c: [0.05, 0.05], r: 0.04 }, { c: [-0.02, 0.075], r: 0.038 }],
  [{ c: [0, 0], r: 1 }, { c: [0, 0], r: 1 }, { c: [0, 0], r: 1 }], // the last bite: all of it
];
// …
mat.maskNode = Fn(() => {
  // Keep a fragment only if it's outside every bite circle.
  const outside: TslNode<"bool">[] = circles.map(({ c, r }) =>
    length(positionLocal.xz.sub(c.mul(scale).add(vec2(offset.x, offset.z)))).greaterThan(r.mul(scale)),
  );
  return outside.reduce((a, b) => a.and(b));
})();

The last bite is three circles with a radius of a metre each, on a sandwich fourteen centimetres long. It takes the lot.

What I'd do differently

Textures are WebP because the tool for KTX2 isn't installed yet. WebP is small on the wire but decompresses to full size in the graphics memory, and KTX2 would stay compressed on the GPU, which matters more on a phone than the download does. I'd also generate concepts from more than one angle for anything the camera sees from below. The preamble asks for a view from slightly above, so undersides are the generator's best guess, and a best guess is exactly what a three-year-old's eye level finds.

The people are next. They go through the same optimiser at the end of a much longer pipeline, from photos, and that's its own page: seven ages on one skeleton.