Performance, Power, and Architectural Analytics

Introduction

Processor evaluation is not simply a search for the highest clock frequency. Real performance depends on instruction count, cycles per instruction, memory behavior, parallelism, software, and workload characteristics. A processor that completes a benchmark quickly may consume excessive energy, overheat in an embedded enclosure, or perform poorly on applications not represented by the benchmark.

Architectural analytics connects measurements to engineering decisions. CPU-time equations explain execution time; CPI and IPC expose pipeline behavior; speedup laws identify optimization limits; roofline analysis relates computation to memory bandwidth; and profiling tools locate bottlenecks. Power, energy, temperature, cost, and reliability then determine whether performance is sustainable and economically useful.

Learning Objectives

After studying this chapter, students should be able to:

• apply CPU-time, CPI, IPC, throughput, latency, and speedup equations;
• decompose CPI using correctly defined event frequencies and penalties;
• use Amdahl’s law and roofline reasoning to evaluate optimizations;
• design defensible benchmark and profiling experiments;
• interpret hardware performance counters without confusing correlation with causation;
• calculate power, energy, thermal, and performance-per-watt measures;
• evaluate DVFS, cost, reliability, and thermal trade-offs; and
• identify misleading or unethical benchmark practices.

1. Defining Processor Performance

1.1 Latency and throughput

Latency, or response time, is the time required to complete one task. Throughput is the number of tasks completed per unit time. Reducing task latency may increase throughput, but the two are not identical. A four-core processor might complete four independent jobs concurrently without making any individual job four times faster.

For a fixed task:

Thus, if machine A completes a task in time T_A and machine B in T_B, the performance speedup of A over B is:

This comparison is valid only when both systems perform equivalent work and produce acceptably equivalent results.

1.2 Elapsed time and CPU time

Elapsed time includes CPU execution, operating-system delays, input/output, communication, and waiting. CPU time measures time spent executing on the processor and may be divided into user and system CPU time. Reporting CPU time alone can hide storage or network bottlenecks; reporting elapsed time alone can include unrelated system activity. The chosen measure must match the engineering question.

2. CPU-Time Equation, CPI, and IPC

A program executes a dynamic instruction count IC. If the average number of processor cycles per retired instruction is CPI and the clock frequency is f, then:

Here, T_CPU is CPU execution time in seconds, IC is the number of retired instructions, CPI has units cycles/instruction, t_cycle is seconds/cycle, and f is cycles/second, or hertz. Dimensional cancellation gives seconds.

The equation is an accounting identity when instruction count, cycles, and CPU time refer to the same execution interval. It does not imply that changing frequency leaves CPI or instruction count unchanged. Higher frequency can alter memory penalties measured in cycles, while compiler changes can alter both instruction count and CPI.

2.1 Weighted average CPI

If instruction class i has count IC_i and average CPI CPI_i, then total cycles and average CPI are:

Equivalently, if p_i = IC_i/IC is the dynamic fraction of class i, then CPI_avg = ∑_i p_iCPI_i, provided that ∑_i p_i = 1.

Worked example: weighted CPI

Suppose 60% of instructions require 1 cycle, 20% require 2 cycles, and 20% require 3 cycles. Then:

The corresponding average IPC is:

The reciprocal relationship holds when CPI and IPC use the same instruction and cycle counts over the same interval. On multicore systems, per-core and aggregate IPC must not be mixed.

2.2 CPI stall decomposition

A useful model separates ideal or base execution from stalls:

In this expression, r_k must be the number of stall-causing events of type k per retired instruction, with units events/instruction. The penalty p_k is the average additional cycles/event. Their product therefore has units cycles/instruction.

If a published miss rate is instead misses per access, it must be multiplied by accesses per instruction:

Likewise:

These additive models assume that penalties have been defined to account appropriately for overlap. On out-of-order processors, cache misses and other stalls may overlap; adding raw penalties can therefore overestimate total CPI.

Worked example: stall CPI

Consider a processor with base CPI 1.00 and the following measured behavior:

• instruction-cache misses: 0.015 misses/instruction, penalty 20 cycles/miss;
• data accesses: 0.30 accesses/instruction, miss rate 0.04 misses/access, penalty 30 cycles/miss;
• branches: 0.20 branches/instruction, misprediction rate 0.05 mispredictions/branch, penalty 12 cycles/misprediction.

The contributions are:

Hence:

If the program retires 2 × 10⁹ instructions at 2.5 GHz, its modeled CPU time is:

3. Speedup and the Limits of Optimization

3.1 Amdahl’s law

Suppose a fraction F of the original execution time is improved by a factor S, while the remaining fraction is unchanged. The overall speedup is:

Amdahl’s law assumes a fixed workload and that the fractions are measured on the original execution. As S → ∞, the maximum speedup approaches 1/(1−F).

Example: If 40% of execution is accelerated fivefold:

Even a powerful accelerator gives only about 1.47× overall speedup because 60% remains unchanged.

3.2 Scaled workloads

Amdahl’s law addresses fixed-size workloads. In parallel computing, users may increase problem size as resources increase. Gustafson’s scaled-speedup model is:

Here, N is the processor count and α is the serial fraction measured in the parallel execution time. The model does not eliminate communication, synchronization, or load-imbalance costs; these must be reflected in the measured serial or overhead component.

4. Roofline Analysis

The roofline model estimates attainable computational performance from peak arithmetic capability and memory bandwidth. Arithmetic intensity I is the number of useful operations performed per byte transferred across the memory boundary being studied.

Here, P_attainable and P_peak are in operations/second, B_mem is bytes/second, and I is operations/byte. The product B_memI therefore has units operations/second.

The ridge point is:

A kernel left of the ridge point is bandwidth-bound under the model; one right of it is bounded by peak compute capability. Real performance may lie below both roofs because of instruction dependencies, poor vectorization, latency, cache conflicts, synchronization, or insufficient parallelism.

Figure: Roofline model separating bandwidth-bound and compute-bound operating regions.

Worked example: A processor has peak performance of 200 GFLOP/s and sustained memory bandwidth of 40 GB/s. Its ridge point is 200/40 = 5 FLOP/byte. A kernel with intensity 3 FLOP/byte has a roofline bound of:

The result assumes consistent decimal units and that 40 GB/s is the relevant sustained bandwidth. Changing the measurement boundary—for example, DRAM versus a cache—changes both traffic and intensity.

5. Benchmarking and Experimental Method

A benchmark is useful only if it represents the intended workload and is measured reproducibly. Common forms include microbenchmarks, application kernels, complete applications, and standardized suites.

A defensible experiment should specify:

1. processor, memory, cooling, firmware, and operating-system configuration;
2. compiler version, flags, libraries, and numerical precision;
3. workload inputs and correctness criteria;
4. warm-up procedure, run count, and treatment of outliers;
5. frequency, turbo, thermal, and power-limit settings; and
6. whether results report latency, throughput, energy, or another metric.

5.1 Aggregating benchmark results

Normalized performance ratios are often aggregated with the geometric mean. For n positive ratios r_i:

The geometric mean treats reciprocal ratios consistently and is less sensitive to the choice of reference machine than the arithmetic mean. It does not indicate absolute execution time and must not replace workload-specific analysis. A weighted aggregate is justified only when the weights represent an explicitly defined usage distribution.

Multiple runs should be reported with variation, such as standard deviation or a confidence interval, when nondeterminism is material. Statistical precision cannot compensate for an unrepresentative workload.

5.2 Common benchmark errors

Misleading conclusions arise from cherry-picking favorable tests, comparing unequal quality settings, using different compiler effort, ignoring thermal throttling, reporting peak rather than sustained results, or hiding failed workloads. A benchmark score is evidence about a defined test—not universal proof that one architecture is superior.

6. Profiling and Hardware Performance Counters

Profiling identifies where execution time and hardware resources are consumed. Instrumentation inserts measurement code; sampling periodically records execution state. Instrumentation can provide exact counts but may disturb timing. Sampling has lower overhead but introduces statistical uncertainty.

Modern processors provide counters for cycles, retired instructions, cache events, branch events, pipeline slots, and other microarchitectural behavior. Basic derived metrics include:

Counters require careful interpretation. Events may be speculative or retired, counter capacity may force multiplexing, and some events are model-specific. Sampling may exhibit skid, where the recorded instruction is near rather than exactly at the event. Virtual machines, interrupts, simultaneous multithreading, and frequency changes can further complicate attribution.

A disciplined workflow is:

1. establish a reproducible baseline;
2. identify time-consuming functions or phases;
3. formulate a bottleneck hypothesis;
4. select counters that can test that hypothesis;
5. change one important factor;
6. verify correctness, speed, energy, and side effects.

Counter values should support causal reasoning rather than encourage indiscriminate collection. For example, a high cache-miss count may be harmless if misses overlap effectively, while a smaller number of serialized misses may dominate execution.

7. Power, Energy, and DVFS

Power is the rate of energy consumption:

Therefore:

For approximately constant average power:

Power is measured in watts, energy in joules, and time in seconds.

7.1 Dynamic and static power

A widely used first-order CMOS switching-power model is:

Here, α is the activity factor, C_eff is effective switched capacitance in farads, V is supply voltage, and f is frequency. The expression is an approximation for capacitive switching; short-circuit and leakage effects require additional terms.

Static power is approximately:

Leakage current depends strongly on process, voltage, and temperature. Total processor power also includes clock networks, memories, interconnects, voltage-conversion losses, and uncore components.

7.2 Dynamic voltage and frequency scaling

DVFS changes voltage and clock frequency to trade performance against power. Lowering frequency alone tends to reduce dynamic power approximately linearly, while lowering voltage can produce a larger reduction because of the V² term. However, voltage cannot generally be varied independently of timing requirements.

Energy does not always decrease when frequency decreases. A longer runtime allows leakage and platform power to accumulate. Conversely, a high-performance “race-to-idle” policy may save energy if the system quickly enters a low-power state. The correct decision requires measured workload and idle-state behavior.

Worked example: performance and energy

An original implementation consumes 20 W for 5 s, so it uses 100 J. An optimized implementation consumes 28 W for 3 s, so it uses 84 J. Performance improves by 5/3 ≈ 1.667, average power rises by 40%, but energy decreases by 16%.

The energy-delay product is:

The original EDP is 100 × 5 = 500 J·s, while the optimized EDP is 84 × 3 = 252 J·s. EDP is useful when both time and energy matter, but its weighting of delay is a policy choice rather than a physical law.

8. Thermal Behavior and Reliability

At thermal steady state, a simplified single-resistance model gives:

Here, T_j is junction temperature, T_a is ambient temperature, P is average dissipated power, and R_θJA is junction-to-ambient thermal resistance in °C/W. The model assumes steady power, a specified package and airflow configuration, and approximately linear heat transfer. Transient behavior requires thermal capacitance or a more detailed thermal network.

For T_a = 35°C, P = 18 W, and R_θJA = 2.5°C/W:

High temperature may trigger throttling and accelerate mechanisms such as electromigration and transistor aging. Rapid thermal cycling can stress interconnects and packaging. Reliability therefore depends on temperature history, voltage stress, workload, manufacturing variation, and cooling—not only average power.

Availability is commonly expressed as:

Here, MTBF is mean time between failures and MTTR is mean time to repair, using consistent time units. This repairable-system metric assumes meaningful long-run averages and does not by itself model correlated or catastrophic failures.

9. Cost and Multiobjective Evaluation

Architectural decisions are usually multiobjective. Relevant measures include:

• performance per watt;
• energy per task;
• performance per unit purchase cost;
• total cost of ownership, including electricity and cooling;
• area efficiency for silicon-limited designs;
• deadline compliance and quality of service; and
• reliability or availability under expected conditions.

Performance per watt is meaningful only when performance and power refer to the same workload, interval, and system boundary. Chip-only power should not be compared with wall-socket system power without explicit normalization.

Pareto analysis is useful when no design dominates all objectives. A design is Pareto-dominated if another design is at least as good in every relevant objective and strictly better in at least one.

10. Illustrative Case: Edge-Vision Processor Selection

Background

This fictional case concerns an engineering team selecting a processor for a battery-backed industrial camera. The device must analyze one frame every second. Both candidates satisfy the same accuracy requirement.

Problem

Candidate A uses CPU processing. Candidate B adds an NPU and costs $8 more per device. The system consumes 0.30 W while idle.

Analysis

Energy per one-second period is active energy plus idle energy.

For Candidate A:

For Candidate B:

Although Candidate B has higher active power, it uses 40.6% less energy per frame-period:

Its active throughput is 4 frames/s, compared with approximately 1.389 frames/s for Candidate A. Active performance per watt is therefore about 0.571 frames/s/W for B and 0.309 frames/s/W for A.

At continuous one-frame-per-second operation, the average-power difference is 1.349 W. Over 8760 hours, Candidate B saves approximately:

Approach and Outcome

The team should validate both candidates with sustained ambient-temperature tests, identical image streams, full-system wall power, latency percentiles, and output accuracy. If electricity, battery size, cooling, or thermal reliability is important, Candidate B’s extra purchase cost may be justified. The break-even calculation should use local energy prices and deployment lifetime rather than processor energy alone.

Learning Points

• Active power is not the same as energy per completed task.
• Accelerators are valuable only when enough workload can use them.
• Idle power and duty cycle can reverse conclusions based on peak power.
• Accuracy, sustained thermals, software support, and cost must accompany performance results.

11. Contemporary Developments and Responsible Practice

Heterogeneous processors increasingly combine CPUs, GPUs, vector units, and AI accelerators. Chiplets and advanced packaging alter bandwidth, thermal density, yield, and cost trade-offs. Machine-learning workloads also require metrics beyond raw operations per second: model accuracy, precision, sparsity assumptions, memory traffic, batch size, and tail latency can change conclusions substantially.

Responsible reporting requires transparent configurations, reproducible methods, equivalent correctness criteria, and disclosure of tuning. Security mitigations should not be disabled silently to improve scores. Energy and carbon claims should state system boundaries and distinguish operational energy from embodied manufacturing impacts. Performance engineering becomes ethically problematic when selective reporting misleads purchasers, regulators, or the public.

12. Best-Practice Checklist

• Define the workload, correctness criterion, and system boundary first.
• Report both latency and throughput when both matter.
• Keep instruction, cycle, and counter intervals consistent.
• Convert miss rates to events per instruction before computing stall CPI.
• Use sustained rather than advertised peak measurements.
• Profile before optimizing and verify after every change.
• Measure energy per task as well as average power.
• Check thermal steady state and throttling.
• Report variation and all important configuration details.
• Treat benchmark conclusions as workload-specific evidence.

Chapter Summary

Processor performance is determined jointly by instruction count, CPI, and clock frequency. IPC provides a complementary view, while properly constructed CPI decompositions connect performance losses to event frequencies and penalties. Amdahl’s law reveals fixed-workload speedup limits, and the roofline model distinguishes memory-bandwidth constraints from compute limits.

Reliable evaluation requires representative benchmarks, reproducible configurations, statistical care, profiling, and cautious use of performance counters. Power and energy are distinct: a high-power processor can use less total energy if it finishes sufficiently faster. DVFS, leakage, idle states, temperature, cost, and reliability must therefore be evaluated together. Sound architectural decisions use transparent, workload-specific, multiobjective evidence rather than isolated peak scores.

Key Terms

Latency: Time required to complete an individual task.

Throughput: Number of tasks completed per unit time.

Instruction count: Number of dynamic instructions retired by a program.

CPI: Average processor cycles per retired instruction.

IPC: Average retired instructions per processor cycle.

Speedup: Ratio of old execution time to new execution time for equivalent work.

Amdahl’s law: Fixed-workload model limiting speedup by the unaccelerated fraction.

Arithmetic intensity: Useful operations performed per byte transferred across a defined memory boundary.

Roofline model: Performance bound combining peak computation and sustained memory bandwidth.

Ridge point: Arithmetic intensity at which compute and bandwidth rooflines intersect.

Benchmark: Workload used to measure and compare system behavior.

Profiling: Measurement process used to locate time and resource consumption.

Performance counter: Hardware register that counts selected microarchitectural events.

MPKI: Cache misses per thousand retired instructions.

DVFS: Dynamic adjustment of processor voltage and frequency.

Dynamic power: Power primarily associated with circuit switching activity.

Leakage power: Power consumed by leakage currents even without useful switching.

Energy-delay product: Product of task energy and execution time.

Thermal resistance: Temperature rise per unit dissipated power under specified conditions.

Pareto optimality: Condition in which no objective can be improved without worsening another.

Review Questions

Short Answer Questions

1. Distinguish processor latency from throughput.
2. State the CPU-time equation and define its variables.
3. Why must a cache miss rate per access be converted before use in a CPI decomposition?
4. What does the roofline ridge point represent?
5. Why can reducing clock frequency sometimes increase task energy?

Long Answer Questions

1. Explain how instruction count, CPI, and frequency interact, including why changing one may affect the others.
2. Derive Amdahl’s law and discuss its assumptions and limitations.
3. Explain how to design a reproducible processor benchmark experiment.
4. Discuss the interpretation problems associated with hardware performance counters.
5. Analyze the relationships among dynamic power, leakage power, energy, temperature, and reliability.

Analytical / Application Questions

1. A program executes 4 × 10⁹ instructions with CPI 1.5 at 3 GHz. Calculate its CPU time. If an optimization reduces instruction count by 20% but raises CPI to 1.7, determine the new CPU time and speedup.
2. A kernel runs on a processor with peak performance 600 GFLOP/s and sustained bandwidth 75 GB/s. Find the ridge point and roofline bound at 4 FLOP/byte. Classify the kernel under the roofline model.
3. Compare two embedded processors using measured latency, active power, idle power, thermal limits, acquisition cost, and expected duty cycle. Construct a justified decision criterion rather than selecting solely by peak performance.

Multiple Choice Questions with Answers

Question 1

A program retires 2 × 10⁹ instructions with CPI 1.5 on a 3 GHz processor. What is its CPU execution time?

A. 0.50 s

B. 1.00 s

C. 1.50 s

D. 3.00 s

Correct Answer: B. 1.00 s

Question 2

An instruction mix contains 60% instructions with CPI 1, 20% with CPI 2, and 20% with CPI 3. What are the average CPI and corresponding IPC?

A. CPI 1.4 and IPC 0.714

B. CPI 1.6 and IPC 0.625

C. CPI 1.8 and IPC 0.556

D. CPI 2.0 and IPC 0.500

Correct Answer: B. CPI 1.6 and IPC 0.625

Question 3

A program performs 0.25 data accesses per instruction. Its cache miss rate is 0.08 misses per access, and each miss adds 40 stall cycles. What is the data-cache stall CPI?

A. 0.08

B. 0.32

C. 0.80

D. 3.20

Correct Answer: C. 0.80

Question 4

Forty percent of original execution time is accelerated by a factor of 5. According to Amdahl’s law, what is the overall speedup?

A. 1.25×

B. 1.47×

C. 2.00×

D. 3.40×

Correct Answer: B. 1.47×

Question 5

A processor has peak performance of 300 GFLOP/s and sustained memory bandwidth of 60 GB/s. What is its roofline ridge point?

A. 2 FLOP/byte

B. 5 FLOP/byte

C. 18 FLOP/byte

D. 360 FLOP/byte

Correct Answer: B. 5 FLOP/byte

Question 6

On the processor with a 300 GFLOP/s peak and 60 GB/s bandwidth, what is the roofline bound for a kernel with arithmetic intensity 3 FLOP/byte?

A. 20 GFLOP/s

B. 60 GFLOP/s

C. 180 GFLOP/s

D. 300 GFLOP/s

Correct Answer: C. 180 GFLOP/s

Question 7

Which expression correctly calculates branch stall CPI when the misprediction rate is stated per branch?

A. Branches per instruction × mispredictions per branch × cycles per misprediction

B. Instructions per branch × cycles per misprediction

C. Mispredictions per branch ÷ cycles per misprediction

D. Branches per cycle × clock frequency

Correct Answer: A. Branches per instruction × mispredictions per branch × cycles per misprediction

Question 8

An implementation consumes 24 W for 4 s. How much energy does it use if average power is constant?

A. 6 J

B. 20 J

C. 28 J

D. 96 J

Correct Answer: D. 96 J

Question 9

Using T_j = T_a + PR_θJA, what junction temperature results from an ambient temperature of 30°C, power of 16 W, and thermal resistance of 2°C/W?

A. 32°C

B. 46°C

C. 62°C

D. 92°C

Correct Answer: C. 62°C

Question 10

Which benchmarking practice provides the most defensible processor comparison?

A. Selecting only workloads on which the preferred processor wins

B. Comparing peak frequency while ignoring software and memory configuration

C. Using equivalent workloads, disclosed configurations, repeated runs, and correctness checks

D. Disabling security mitigations on only one processor

Correct Answer: C. Using equivalent workloads, disclosed configurations, repeated runs, and correctness checks