Hardware & computer systems
Move the data. Meet the physical budget.
Translate a real workload into compute, memory, interconnect, I/O, power, thermal, and reliability decisions—then prove the whole machine meets its deadline.
- 15design steps
- 12bottlenecks
- 1worked interview
# Workload before components
ops = profile(shape, reuse, precision)
deadline = define(latency, throughput, jitter)
movement = count(bytes across every boundary)
design = choose(compute + memory + fabric)
verify(design, power, thermal, cost, reliability)
while not all_satisfied(measurements, limits):
diagnose(the limiting constraint)
change_one_variable()
remeasure()01 · Workload-to-architecture flow
Architecture is a chain of justified allocations
Do not begin with a favorite processor. Each step produces evidence for the next one.
- 01Characterize the workload
Operators, tensor or packet shapes, control flow, reuse, sparsity, precision, working set, and concurrency.
Output: representative kernels + traces - 02Set measurable targets
Tail latency, sustained throughput, jitter, boot time, availability, lifetime, and accepted error.
Output: pass/fail requirements - 03Count data movement
Bytes read and written at registers, caches, SRAM, DRAM, storage, network, and device boundaries.
Output: bandwidth + capacity budgets - 04Expose parallelism
Task, data, pipeline, vector, and request-level parallelism; identify serial regions and dependencies.
Output: useful—not theoretical—parallelism - 05Choose compute
Allocate work across CPU, GPU/NPU, FPGA, fixed-function engines, and firmware based on regularity and change rate.
Output: compute partition - 06Build the memory + fabric
Place data by reuse and lifetime; size buffers; choose coherency, NoC, external links, and flow control.
Output: data path + topology - 07Close physical budgets
Estimate PPA, cost, power delivery, cooling, signal integrity, reliability, manufacturing, and serviceability.
Output: feasible implementation - 08Plan proof + observability
Trace requirements, interfaces, and faults to pre-hardware evidence, first-article checks, qualification, and production test; reserve trace, JTAG, BIST, loopback, test points, counters, and known-good signatures.
Output: verification + bring-up plan - 09Implement + sign off
For custom logic, close RTL, CDC/RDC, coverage, timing, power, DFT, and physical sign-off. For board/COTS, close schematic/layout, SI/PI/thermal, DRC/DFM, firmware compatibility, BOM, and vendor pedigree.
Output: controlled build package - 10Bring up
Prove first hardware in dependency order, correlate measurements with the model, localize faults with planned hooks, and record errata.
Output: characterized first article - 11Verify requirements
Trace each hardware, interface, timing, power, reliability, security and recovery requirement to objective evidence on the controlled configuration.
Output: compliance + anomaly record - 12Validate intended use
Exercise the complete product with representative operators, workloads, dependencies, environments and failure responses; measure mission outcomes and limitations.
Output: stakeholder-use evidence - 13Qualify configuration
Bound environmental, EMC, performance, reliability, lifetime and interoperability limits for the exact design and supported hardware/software configuration.
Output: qualified operating envelope - 14Accept + release
Define production/lot/unit screening, calibration, secure provisioning, traceability and acceptance limits separately from design qualification; stage release with rollback.
Output: accepted release baseline - 15Sustain + retire
Use telemetry, RMA and failure analysis to control fixes, updates, obsolescence, service, deprovisioning, data disposition, and end of life.
Output: field evidence + retirement record
Memory hookWorkload → deadline → bytes → parallelism → compute → memory/fabric → physics → proof → sign-off → bring-up → verify → validate → qualify → accept/release → sustain.
02 · Performance contract
Latency, throughput, and bandwidth are related—not interchangeable
A design can finish one request quickly and still fail sustained load. It can have enormous compute and still wait for bytes.
Concurrencyconnects response time to work in flight
Overlap stages so multiple items are in flight. In a dedicated, balanced pipeline, the slowest stage bounds steady-state rate; shared resources, bubbles, variability, and initiation interval can lower it further. Pipeline depth may increase single-item latency.
Shorten serial dependencies, avoid queueing, use nearby storage, and reserve capacity for tail behavior—not only the average.
Account for protocol overhead, access efficiency, bank conflicts, contention, and read/write turnaround. Peak pin rate is not delivered payload.
average work in flight = average effective throughput × average time in system, measured over the same stable boundary. Do not multiply peak throughput by p99 latency and call the result Little’s Law.
Interview move: name the unit and percentile every time—“30 fps sustained” and “p99 under 80 ms” are testable; “fast” is not.
03 · Physical trade-off compass
Performance lives inside power, area, and cost
Optimizing one corner usually spends another. State which budget is fixed and which may move.
IPC, frequency, parallel units, utilization, memory service time, and tail behavior.
Measure real workloadsDynamic switching, leakage, voltage rails, transient demand, battery life, and energy per task.
Average + peak both matterCompute lanes, SRAM, PHYs, package pins, PCB layers, routing pressure, and yield.
Idle silicon is still paid forNRE, masks, package, memory, board, cooling, test time, yield, supply risk, and field service.
Bill of materials is one slice
More parallel units↔area + leakage + routing↔higher memory demand↔power + thermal + cost
04 · Memory hierarchy
Place data by reuse, lifetime, sharing, and control
Local resources often reduce access latency and energy, but caches, scratchpads, shared SRAM, DRAM, and storage are not one universal ladder. Compare management, visibility, capacity, access shape, coherence, and measured cost on the target architecture.
- Registerscompiler allocated · lane / thread scope · operands
- Hardware-managed cachesprivate or shared · L1 / L2 / LLC topology varies
- Local scratchpad / shared memorysoftware or compiler managed · core / SM / tile scope
- Shared on-chip SRAM / system cachecluster or accelerator scope · banks, arbitration, ownership
- Off-chip DDR / HBMdevice or system scope · controllers, channels, rows, ECC
- Persistent and remote storesNVMe or network I/O branch · separate latency, durability, failure contracts
Include code, metadata, double buffers, fragmentation, ECC, reserved memory, and worst-case concurrency.
Tile loops and tensors, fuse operators, batch carefully, and keep producer output near the next consumer.
Bursts favor wide transfers; random small requests expose bank, row, translation, and command overhead.
Choose coherent caches when shared-memory convenience earns its traffic; use message passing or scratchpads for explicit control.
Memory-level parallelism hides service time only when dependencies, buffers, and controllers allow it.
Plan IOMMU/MPU domains, address translation, secure memory, ECC/parity, and scrub or reset behavior.
Roofline reasoning
Ask whether the kernel is byte-bound or compute-bound
Arithmetic intensity is useful operations divided by bytes moved at the memory boundary being analyzed. The attainable rate is bounded by the lower of compute peak and bandwidth × intensity.
- Left slope
- Increase reuse, coalesce traffic, compress, or add bandwidth.
- Flat roof
- Improve utilization, vectorization, mapping, frequency, or compute resources.
- Below both
- Look for serialization, imbalance, instruction overhead, synchronization, or tiny problem sizes.
The cyan bandwidth slope rises from the lower left to a knee where it meets the flat amber compute ceiling. The streaming-kernel point lies below the slope in the memory-bound region; the reused-tile point lies below the flat ceiling after the knee.
Architecture example: NVIDIA documents configurations in which L1 data cache and shared memory use one unified resource, illustrating why cache/scratchpad ordering and capacity must be target-specific. CUDA programming model ↗
05 · Compute and dataflow choices
Match programmability to regularity
The best engine is the least specialized one that still meets the whole-system target.
Compute-engine selection tree
Use the least specialized engine that closes the whole path
This chooses the primary engine for a measured hotspot—not one engine for the whole computer.
Gate 01 · general-purpose firstCan a CPU meet latency, throughput, energy, and cost targets with margin?Include I/O, operating-system work, memory traffic, contention, and tail or worst-case behavior under representative load.
Keep irregular, branch-heavy, latency-sensitive control/orchestration, sparse, and frequently changing work on the CPU when it cannot amortize or justify an offload boundary.
Trade-off: lower throughput per watt on regular parallel kernels.What structure does the hot path expose?
Specialize only after any candidate path is provenStable workload and interfaces + sufficient volume + lifetime energy, area, and unit-cost savings justify NRE and validation?Yes → consider ASIC / fixed function for the proven tensor, streaming, protocol, or control hotspot. No → retain the programmable CPU, GPU/NPU, or FPGA path.
Every leaf converges here
Does the candidate win end to end?
- Movebytes · locality · transfers
- Waitqueues · launch · sync · conversion
- Useutilization · tails · worst case
- Fitpower · thermal · area · cost
- Ownsoftware · debug · update · fallback
No: revisit the partition, dataflow, or engine. Yes: prototype the complete path and verify measured margin.
Keep weights local and stream activations. Useful when weights are heavily reused across batches or spatial positions.
Watch: activation traffic + accumulationKeep partial sums local until complete. Useful when reductions dominate and accumulator movement is expensive.
Watch: accumulator capacity + precisionExploit multiple reuse directions across weights, inputs, and partial sums. Useful when no single tensor dominates movement.
Watch: mapping and control complexitySelection testFor each candidate, estimate useful utilization, bytes moved, synchronization, conversion overhead, fallback work, and software cost.
06 · Interconnect and I/O
The fabric is part of the computer
Topology, coherency, ordering, flow control, DMA, and external links determine whether compute and memory can work together.
CPU clustercontrol · OS · serial work
GPU / NPUparallel kernels
Security enginekeys · boot · isolation
On-chip fabricNoC · coherency · QoS
↔
routes, arbitration, ordering, backpressure, protection, observability
SRAM / DRAMcontrollers · ECC
PCIe / CXL / Ethernetexternal expansion
Display · camera · storagereal-time I/O
Choose from endpoint count, locality, bisection bandwidth, hop latency, layout, and growth.
Define which agents share memory, cache ownership, snoop scope, consistency model, and non-coherent boundaries.
Priorities, virtual channels, credits, admission limits, and reservations can bound interference only when every shared interconnect, bridge, queue, and memory controller honors a compatible policy. Verify worst-case latency and bandwidth under concurrent traffic; a QoS tag alone is not isolation.
The CPU or driver still prepares mappings or descriptors, handles completion and errors, and may perform cache maintenance. It is zero-copy only when producer and consumer safely share the same owned and mapped buffer.
Memory barriers, posted writes, completion queues, atomics, interrupts, and error reporting close the contract.
Lane rate, encoding, packet headers, retransmission, switch hops, connector limits, reach, and signal margin reduce payload rate.
Protocol anchor: Arm defines AXI QoS signaling but leaves system policy to the implementation; verify the complete path. AMBA AXI and ACE specification ↗
07 · Power, thermal, and reliability
A fast block can create a slow system
Power delivery and heat removal determine sustainable performance. Reliability defines whether that performance survives its environment and lifetime.
- 01 · ActivityMore switchingfrequency, voltage, toggles, memory traffic
- 02 · PowerRail demand risesaverage draw + transient droop
- 03 · HeatJunction temperature risespackage, board, airflow, ambient
- 04 · ControlDVFS or throttlingprotect limits; reduce delivered rate
- 05 · OutcomeDeadline or lifetime risktail latency, aging, shutdown, user impact
Regulator response, decoupling, package/board impedance, rail sequencing, brownout detection, and safe reset.
Use steady-state resistance for equilibrium and time-dependent thermal impedance/capacitance for pulse width and duty cycle; include hotspots, sensors, cooling curve, ambient, enclosure, and fan failure.
ECC/parity, CRC, replay, memory scrubbing, poison propagation, watchdogs, and checkpoint/restart.
Process corners, voltage/temperature margin, wear, endurance, component derating, diagnostics, and field update.
08 · Hardware bottleneck selector
Diagnose from counters and behavior
Change one variable at a time. A symptom can have several causes, so confirm the limiting resource before redesigning.
| Bottleneck | Typical symptom | Measure | First useful action |
|---|---|---|---|
| Compute execution throughput | Execution units are saturated; more data bandwidth does not help. | IPC, useful ops/cycle, unit occupancy, instruction mix, dependency stalls. | Increase useful parallelism, vectorize, fuse work, or lower operation count. |
| Serial dependency | Low occupancy despite ready data; one chain controls completion. | Critical path, dependency stalls, Amdahl fraction. | Break dependencies, pipeline stages, speculate, or move serial control to a faster core. |
| Memory bandwidth | Performance scales with memory clock/channels and plateaus. | Delivered bytes/s, controller utilization, arithmetic intensity. | Increase locality, tile/fuse/compress, reduce precision, or add channels. |
| Memory latency | Small random work stalls; bandwidth remains underused. | Outstanding requests, miss latency, row hit rate, queue depth. | Prefetch, batch, reorder, cache indices, add memory-level parallelism. |
| Cache / TLB | Sharp slowdown when working set crosses a size boundary. | Miss rates by level, page walks, conflict/eviction counters. | Change layout, tile, use suitable pages, partition cache, or use scratchpad. |
| On-chip fabric | Remote agents slow each other; hop or arbitration stalls rise. | Link occupancy, hotspots, credits, latency by source/destination. | Place communicating blocks closer, widen links, rebalance routes, add QoS. |
| External I/O | Compute waits for camera, disk, network, or host transfer. | Payload rate, packet size, retries, DMA gaps, link state. | Overlap transfers, batch, share mapped buffers when ownership/coherency permits, compress, or upgrade the link. |
| Synchronization | More cores reduce scaling; lock or barrier time rises. | Lock wait, atomic retries, barrier skew, coherence invalidations. | Partition ownership, reduce shared writes, shard locks, use message passing. |
| Load imbalance | Some engines idle while one queue or tile is full. | Per-engine utilization, queue depth, task duration distribution. | Change partitioning, use work stealing, dynamic tiles, or split long tasks. |
| Conversion / formatting | Copy, layout, precision, or protocol conversion dominates. | Time and bytes in adapters; accelerator-to-accelerator gaps. | Standardize layouts, fuse conversion, support the format in hardware. |
| Power / thermal | Fast initially, slower after seconds or minutes. | Rail power, clocks, temperature, throttle residency. | Reduce movement/switching, improve cooling, schedule bursts, resize peak. |
| Reliability margin | Errors appear at voltage, temperature, frequency, or age corners. | ECC events, retries, droop, timing margin, fault logs. | Add detection/recovery, margin critical paths, derate, isolate faulty resources. |
1 · Reproducerepresentative workload + steady state
2 · Observetime, counters, traces, power, temperature
3 · Perturbchange one clock, channel, batch, or mapping
4 · Modelpredict the improvement if the hypothesis is true
5 · Verifymeasure gain and check the next constraint
09 · Quantitative sizing workbench
Follow bytes, operations, and heat together
Peak TOPS is only one ceiling. A defensible hardware proposal estimates the full data path, states the operation-count convention, budgets sustained utilization, and checks the steady thermal state.
Find the first resource that can invalidate the concept
Assume four 1920 × 1080 cameras at 30 frames/s. For the worked DRAM calculation, assume the ISP produces tightly packed 8-bit YUV 4:2:0 surfaces at 1.5 bytes per active pixel; budget sensor-link ingress separately from the selected RAW or YUV wire format and its overhead. The illustrative model requires 30 GOp per camera-frame, where 30 billion operations are already counted under the convention one MAC = two operations. Treat every value as an input to verify on the selected sensor, ISP, model, surface layout, and counting convention.
- Post-ISP surface rate4 × 1920 × 1080 × 30 ≈ 249 million active pixels/s. At 1.5 bytes/pixel, one aggregate full-frame transfer-equivalent is about 373 MB/s before surface stride, alignment, or metadata. Separately count sensor ingress from the actual RAW/YUV packing, transmitted blanking if any, packet overhead, and error traffic.
- Memory movementIf the ownership ledger totals four aggregate full-frame transfer-equivalents—each one exactly one complete DRAM read or write of all four post-ISP streams—payload traffic is about 1.49 GB/s. Four is illustrative, not one transfer per named stage: draw every producer–consumer edge and count its reads and writes.
- Compute demand30 GOp/camera-frame × 120 camera-frames/s = 3.6 TOPS sustained under the stated convention. At 50% measured end-to-end utilization, 7.2 peak TOPS is the arithmetic minimum implied by that measurement—not the provisioned target. Add a named margin for utilization uncertainty, tail scenes, model growth, telemetry, and fallback. If the input were 30 GMAC/frame instead, the corresponding minima would be 7.2 TOPS sustained and 14.4 peak TOPS before that margin.
- Locality testTile the pipeline so the ISP, accelerator, and encoder reuse line or tile buffers. Keeping one intermediate off DRAM may save more energy and latency than adding another arithmetic unit.
- Steady-state proofMeasure delivered bandwidth, engine occupancy, queue depth, rail power, junction temperature, and throttle residency after thermal equilibrium—not only during a short benchmark burst.
| Boundary | Budget | Evidence to collect | Decision trigger |
|---|---|---|---|
| Sensor ingress | Selected RAW/YUV wire payload, packing, transmitted blanking if any, packet overhead, and error/retry traffic per port and aggregate. | Lane utilization, CRC/error counts, timestamp skew, receiver FIFO high-water mark, dropped frames. | Add lanes, lower format/rate, isolate a faulty stream, or reserve ingress bandwidth. |
| On-chip fabric | Read/write bytes per producer–consumer edge, burst shape, multicast, QoS class, and worst concurrency. | Per-link occupancy, arbitration stalls, credits, hop latency, backpressure, deadline misses. | Change placement/routes, widen a link, reserve service, split traffic classes, or reduce crossings. |
| DRAM | Every read and write, row/bank behavior, refresh, ECC, controller efficiency, and recovery headroom. | Delivered bytes/s rather than pin rate, queue depth, row hits, latency distribution, corrected errors. | Improve locality/layout, add channels, compress, keep intermediates on chip, or reduce precision. |
| Compute engines | Supported useful operations, issue/execute rate, occupancy, synchronization, conversion, and fallback. | Useful ops/cycle, busy versus stalled cycles, critical path, queue gaps, unsupported-op time. | Remap/fuse/vectorize, rebalance engines, change precision, accelerate the serial path, or simplify the model. |
| Power + thermal | Rail-average and transient power, heat path, ambient, fanless enclosure, aging/dirty cooling, and safe limits. | Voltage droop, subsystem energy, junction/board temperature, clock residency, throttle time after soak. | Lower movement/activity, stagger engines, adjust DVFS/admission, improve cooling, or reduce sustained throughput. |
| Fault | Containment | Recovery | Proof |
|---|---|---|---|
| NPU command timeout | Fence admission, mark output invalid, advance the request/completion epoch, preserve fault context, and keep DMA mappings and backing buffers pinned while attempting a proven quiesce. Advance a reset or residency generation only when that separate state transition actually occurs. | Confirm quiescence or contain the device with the documented bus-master-disable, IOMMU, or reset-domain sequence and drain posted writes; only then retire descriptors, unmap/reuse buffers, restore state, run self-test, and resume or enter fallback. | Inject hangs, late completions, and late writes at every command/reset phase; prove no old request epoch, partial output, unmapped buffer, or recycled memory becomes valid. |
| Uncorrectable memory error | Poison the affected line/page, identify consumers, prevent silent propagation. | Retry only where semantics permit, reconstruct replicated state, retire memory, or degrade service. | Error injection from controller through software notification and user-visible behavior. |
| Camera error flood | Bound retries/interrupt rate and isolate its queue so other streams keep their reserved service. | Reinitialize receiver/sensor, rejoin with timestamp resynchronization, or declare the stream unavailable. | Malformed packet and disconnect campaigns at aggregate peak load. |
| Thermal or rail excursion | Throttle or shed workload before violating electrical, temperature, latency, or accuracy limits. | Use hysteresis, staged clocks/power, controlled restart, and a latched service state after repeated failure. | Worst-ambient thermal soak, load steps, brownout, sensor fault, and recovery-loop testing. |
| Interrupted update | Authenticate before activation; keep boot-critical state and rollback metadata power-fail safe. | Boot a compatible signed A/B image, confirm health, enforce anti-rollback policy, and retain a recovery path. | Cut power at every update stage and test old/new firmware, model, schema, and calibration compatibility. |
Budget capacity and traffic
Model weights, worst-live activations, camera buffers, double buffering, descriptors, firmware, OS, fragmentation, ECC, debug capture, and update slots. Capacity and bandwidth are different constraints.
Define ordering and ownership
For every producer–consumer edge state address space, coherency mode, cache maintenance, DMA direction, fences/events, QoS, backpressure, timeout, reset scope, and what happens to in-flight writes.
Match engine to operator
Separate regular dense kernels, vector/reduction work, scalar control, ISP/codec functions, and unsupported fallbacks. Include layout conversion and synchronization in the accelerator boundary.
Close PPA and RAS
Show rail and thermal budgets, voltage/frequency corners, ECC/parity coverage, poison and retry behavior, watchdog/reset hierarchy, diagnostic visibility, aging assumptions, and degraded service.
10 · Realization, bring-up, and release
Prove first hardware in dependency order
Bring-up is a controlled experiment, not “apply power and see.” Define the configuration, instrument, expected signature, stop limit, recovery action, and evidence owner before each step.
- 01PowerInspect assembly and shorts; use current limits; confirm rail value, sequence, ripple, inrush, and idle thermals.
- 02Clock + resetVerify oscillators, PLL lock, reset assertion and release, straps, and required sequencing at each domain.
- 03Debug + bootEstablish JTAG or console, read identity and status, execute BootROM, and load a known-good signed recovery image.
- 04MemoryCheck training, address map, ECC, destructive and retention tests, bandwidth, errors, and temperature sensitivity.
- 05I/OExercise one interface at a time with loopbacks or known peers; inspect signal margin, protocol status, retries, and DMA ownership.
- 06AcceleratorRun a known vector, compare exact or tolerance-bounded output, and test interrupts, timeout, reset, late writes, and fallback.
- 07WorkloadRun the representative concurrent workload through thermal equilibrium; correlate latency, throughput, traffic, power, and temperature with the model.
Custom logic: specification-to-RTL trace, lint, CDC/RDC, assertions, simulation/formal coverage, timing, power, DFT, and physical checks. Board/COTS: controlled BOM and errata, schematic/layout review, SI/PI/thermal, DRC/DFM, firmware/driver compatibility, and vendor qualification evidence.
Run environmental, EMC, interface, security, performance, and reliability campaigns against declared configurations and acceptance criteria. Include applicable architecture/compliance evidence—such as Arm SystemReady for a selected Arm platform—without treating one certificate as whole-system qualification.
Control manufacturing-test coverage and limits, calibration, secure provisioning, serial and lot traceability, golden images, tester versions, yield trends, escape rules, rework, deviations, and release authority.
Use telemetry, RMA and failure-reporting/corrective-action evidence to reopen models, hazards, tests, and baselines. Protect, detect, and recover platform firmware; manage errata, secure updates, component obsolescence, service data, credential deprovisioning, reuse, and disposal.
First-article ruleDo not advance because one output “looks right.” Advance when the declared signature passes, raw evidence is captured, anomalies are owned, and the next step cannot hide an unresolved dependency.
11 · Hardware interview practice
Design an edge AI computer
Speak through the checkpoints before revealing the coaching notes. The goal is a defensible allocation—not a single “correct” chip.
45-minute prompt
Design a fanless roadside perception computer
Process four 1080p camera streams, detect and track objects locally, publish compact events, and retain evidence clips. The unit must operate continuously without cloud inference.
- 4 × 1080p30
- p99 < 100 ms
- 25 W module
- −20–60 °C ambient
- 24 h offline
- 5-year field life
Camerascapture + timestamps
→
Receiver + ISPunpack · demosaic when RAW · resize · shared surfaces
- NPUdetection inference
- CPUtracking + policy
- Networkbounded event queue + health
- Encoderhardware codec when clips are compressed
- Ring buffer + storageevidence clips + retention
Pipeline anchors: GStreamer tee + queue behavior ↗ and NVIDIA DeepStream reference pipeline ↗.
00–07 minClarify the contract Ask about model accuracy, stream format, simultaneous frames, event rate, retention, safety impact, security, environment, size, cost, and field update.
Reveal coaching notes
Separate perception latency from end-to-end event latency. Ask whether dropped frames are allowed, whether all cameras must be synchronized, and what degraded behavior is acceptable when a sensor or accelerator fails.07–15 minBuild workload + movement budgets Estimate decoded pixels, tensor bytes, operations per frame, model/activation footprint, storage/day, network payload, and peak concurrency.
Reveal coaching notes
Four 1080p30 streams are about 249 million pixels/s before format factors. Do not route every full frame through general DRAM repeatedly: use the ISP, tiled buffers, producer-consumer pipelines, and zero-copy surfaces where possible.15–25 minAllocate compute and memory Choose engines for sensor receive/ISP, any actual compressed-video decode, inference, tracking, control, encryption, and storage; size DRAM and on-chip buffers.
Reveal coaching notes
Use the camera receiver and fixed-function ISP for stable unpacking, demosaic only when the sensor source is Bayer/RAW, color conversion, and resize operations; use a codec only if the input is compressed. Map supported model layers to an NPU and keep CPU/vector fallback for control and unsupported operators. Preserve headroom for model growth, telemetry, and failover—not only benchmark throughput.25–33 minDraw the fabric and I/O Place camera inputs, memory controllers, compute, storage, network, DMA, coherency boundaries, QoS, clocks, and reset domains.
Reveal coaching notes
Reserve fabric and DRAM service for camera capture so inference bursts cannot drop frames. Define buffer ownership and completion explicitly. Isolate the management/security plane from bulk video traffic.33–40 minClose physical + failure budgets Explain sustained 25 W operation, thermal path, brownouts, watchdog/reset, ECC, storage wear, sensor loss, NPU hang, network loss, and safe update.
Reveal coaching notes
Model the worst ambient and dirty or aging cooling path. Use staged recovery: restart a job, reset the engine, restart the pipeline, then reboot. Keep a local health channel and signed A/B firmware/model images.40–45 minProve, validate, qualify, and release Name the evidence ladder from pre-hardware proof through first article, requirement verification, intended-use validation, design/configuration qualification, production acceptance, field feedback, and retirement—then state the first likely bottleneck.
Reveal coaching notes
Replay timestamped worst-case scenes while measuring p50/p99 latency, dropped frames, DRAM/fabric utilization, power, junction temperature, throttling, and accuracy through thermal equilibrium. For first article, start with current-limited power and rails, then clocks/reset and debug/boot before memory, I/O, accelerator, and workload. Trace compliance to requirements; separately validate representative roadside operations, operators, dependencies, degraded behavior, and mission outcomes. State the qualified environmental/reliability/interoperability envelope; keep production screening, calibration, provisioning and per-unit acceptance distinct. Close with staged release/rollback, field telemetry/RMA and errata handling, secure-update recovery, obsolescence, and deprovisioning at retirement.
Recheck NPU support, model/activation capacity, bandwidth, latency, accuracy, and update time.
Bound retries, isolate its queue, preserve other streams, report health, and degrade explicitly.
Show sustainable clocks and latency under thermal equilibrium and worst enclosure conditions.
Choose reduced-rate CPU inference, simpler model, sensor-only recording, or declared loss of function.
Strong close“I chose this partition because measured operations and bytes fit with headroom; these are the physical limits; this is how the system degrades and how I will verify it.”
12 · Primary references
Deepen the models behind the diagrams
Use vendor and project documentation for interfaces; use laboratory work for performance models; verify every estimate on the target platform.
Arithmetic intensity, bandwidth ceilings, compute ceilings, and bottleneck-oriented performance analysis.
Berkeley Lab guide ↗Official architecture family for AXI, CHI, AHB, APB, coherency, and SoC component communication.
Arm AMBA specifications ↗CPU, physical, bus, and IOMMU addresses; coherent versus streaming mappings; buffer lifecycle.
Kernel documentation ↗Primary specification hub for PCI Express architecture and related interconnect technologies.
PCI-SIG ↗Open ISA specifications and profiles useful for reasoning about the hardware/software contract.
RISC-V ratified specifications ↗Continue from workload and memory reasoning into analog-aware compilation, mapping, calibration, and runtime design.
Open the ACiM track →Williams, Waterman, and Patterson’s model for relating arithmetic intensity to compute and memory-bandwidth ceilings.
Read via DOI ↗Channels, transactions, ordering, barriers, coherency extensions, atomic behavior, and the contracts connecting processors, memory, and accelerators.
Arm specification PDF ↗Cross-device buffer sharing, reservation objects, DMA fences, explicit synchronization, and the deadlock risks around indefinite dependencies.
Kernel documentation ↗A concrete vendor reference showing channelized HBM connectivity, NoC integration, capacity, and system-level bandwidth considerations.
AMD technical reference ↗
BackendDistributed softwareAPIs, data, caching, queues, reliability.
Systems engineeringWhole lifecycleNeeds, requirements, interfaces, risk, V&V.
EmbeddedCyber-physical systemsReal time, control, power, safety, verification.
NPU + ACiMHardware/software co-designCompiler, mapper, runtime, calibration.
AtlasChoose by system boundaryReturn to the visual track selector.
DSA AtlasAlgorithms and data structuresReturn to the original visual learning map.