Lakehouse Prepchief architect

#Whiteboard Prompts

Thirty things to draw cold, on paper, against a timer. Drawing is different from knowing. You discover what you actually understand at the moment you have to place the first box.

Protocol: 4 minutes to draw, 2 minutes to narrate aloud, no reference material. If you cannot draw it in 4 minutes, you cannot draw it while being interrupted.

#Tier 1 — must be automatic (draw these until they are muscle memory)

  1. Control plane vs compute plane, classic and serverless, showing where customer data lives.
  2. The Delta table on disk — data files plus _delta_log, with a commit and a checkpoint.
  3. The medallion architecture, with what guarantee each layer provides.
  4. Unity Catalog's object hierarchy, from account down to table.
  5. The Spark query lifecycle — SQL text → parse → analyse → optimise → physical plan → tasks.
  6. A job → stages → tasks diagram, showing where the shuffle boundary falls.
  7. A three-environment workspace topology with one metastore and catalog bindings.
  8. The enterprise reference architecture — sources, ingest, layers, consumption, governance spine.
  9. A governed RAG pipeline, showing where permissions are enforced.
  10. Optimistic concurrency control — two writers racing for version N+1.

#Tier 2 — should be fluent

  1. A CDC pipeline from Oracle to governed gold tables, with the audit history preserved.
  2. A regulated deployment with Private Link, no public IPs, CMK and egress control.
  3. A multi-region architecture with data residency, showing what may cross a border.
  4. A DR topology showing what replicates automatically and what must be rebuilt from code.
  5. A streaming pipeline with checkpointing, watermark and state store.
  6. Data mesh on UC — domains, the paved road, and federated governance.
  7. A broadcast join vs a sort-merge join, showing the data movement in each.
  8. A skewed stage, and the same stage after salting.
  9. The medallion layers for a small source where two layers is the right answer.
  10. CI/CD: Git → CI → asset bundle → dev/test/prod, with the service principal boundary.

#Tier 3 — stretch (draw these if the first twenty are automatic)

  1. A real-time serving architecture, two ways: a duplicated serving store vs Lakehouse//RT on governed tables — and the governance difference between them.
  2. Lakehouse Federation as a migration runway, showing consumers cut over before data moves.
  3. Unity Catalog vs Unity AI Gateway — assets versus interactions, on one diagram.
  4. Delta Sharing to three recipient types: another Databricks account, an open-protocol client, and a clean room.
  5. The agent stack: model serving, vector search, retrieval, gateway governance, tracing.
  6. A cost-attribution flow: tags → system tables → showback per domain.
  7. The Hive metastore → UC migration, showing what changes at each step.
  8. Liquid clustering vs Z-order, drawn as file layout and what gets skipped.
  9. A lambda-style architecture and why you would collapse it into one governed pipeline.
  10. The whole platform operating model: who owns what between platform team and domains.

#Narration templates

Drawing without narration is half the drill. Use these fixed orders so you never fumble the opening.

For an architecture diagram:

"I'll start with governance because it's the spine, then work left to right through ingest, the storage layers, and consumption, and finish with how it's operated."

For a mechanism diagram:

"There are three things happening here: [A], [B] and [C]. The important one is [B], because that's where [the property they care about] comes from."

For a comparison diagram:

"Same requirement, two shapes. The difference that matters isn't performance — it's [governance / operational surface / cost shape]. Here's what each costs you."

#Self-check

After each drawing, ask:

  • Could someone build from this, or is it a marketing diagram?
  • Did I draw the data flow and the control/governance flow?
  • Did I label the boundaries — network, account, region, trust?
  • Did I say what I left out and why?

The last one is the senior move. "I've deliberately not drawn the ML platform here, because until the governance layer exists it's premature — that's phase three."