3. Performance Measurement and Scalability Models

Introduction and Context

High-performance computing (HPC) is concerned not merely with making a program run, but with making it run efficiently, predictably, and at an appropriate cost. A program that completes in ten minutes on one processor may appear satisfactory; however, if it takes nine minutes on one processor and eight minutes on sixty-four processors, the additional hardware has delivered little value. Performance measurement therefore provides the evidence needed to distinguish genuine improvement from apparent improvement.

Performance analysis is especially important in data science. Data preparation, simulation, machine learning, graph processing, and scientific analytics often involve large datasets and repeated computation. Their performance depends on processor speed, memory bandwidth, communication, storage, synchronization, algorithmic structure, and system configuration. Consequently, no single metric is sufficient. A meaningful evaluation combines execution time, throughput, latency, floating-point performance, utilization, arithmetic intensity, speedup, efficiency, scalability, and variability.

This chapter develops a framework for measuring and interpreting HPC performance. It introduces the main metrics, explains strong and weak scaling, and examines Amdahl’s Law, Gustafson’s Law, and the roofline model. It also discusses statistically sound benchmarking, practical experimental design, and the organizational decisions that follow from performance evidence.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Distinguish latency, throughput, execution time, FLOPS, utilization, and efficiency.

2. Calculate speedup and parallel efficiency and interpret their limitations.

3. Explain strong scaling, weak scaling, and the factors that limit scalability.

4. Apply Amdahl’s Law and Gustafson’s Law to parallel workload analysis.

5. Use arithmetic intensity and the roofline model to diagnose performance bottlenecks.

6. Design statistically credible HPC benchmarks and report results responsibly.

7. Evaluate performance trade-offs in a practical data-science or HPC scenario.

Performance as a Measurable Property

The first quantity in most performance studies is elapsed or wall-clock time. If a workload starts at time \(t_0\) and completes at \(t_1\), its execution time is

\[T = t_1 - t_0.\]

Lower execution time generally indicates better performance for a fixed task. CPU time, in contrast, measures time spent using a processor and may exclude waiting, communication, input/output, or operating-system activity. Wall-clock time is usually the appropriate measure for user experience and end-to-end service performance, while CPU time is useful for diagnosing computation and resource consumption.

Latency is the time required to complete one operation or respond to one request. It is central to interactive analytics, database queries, and online inference. Throughput is the amount of work completed per unit time, such as records per second, images per second, transactions per second, or jobs per hour. A system can have high throughput but poor latency if it processes many requests in batches. Conversely, a low-latency system may not achieve high aggregate throughput.

For a workload processing \(N\) units in time \(T\), throughput can be expressed as

\[X = \frac{N}{T}.\]

The reciprocal relationship between time per unit and throughput should be interpreted carefully. Improving throughput does not always improve the response time of an individual request, particularly when batching, queueing, or resource sharing is involved.

FLOPS and Related Measures

FLOPS means floating-point operations per second. If an application performs \(F\) floating-point operations in time \(T\), its achieved performance is

\[P = \frac{F}{T}.\]

Common units are MFLOPS, GFLOPS, TFLOPS, and PFLOPS. A processor’s advertised peak FLOPS is a theoretical upper bound based on instruction width, clock frequency, and the number of arithmetic units. Achieved FLOPS is the performance measured for a particular program. The two can differ substantially because of memory stalls, branching, synchronization, insufficient parallelism, and communication.

FLOPS is meaningful mainly for floating-point-intensive workloads. It is less informative for algorithms dominated by integer operations, memory movement, indexing, compression, or input/output. Reporting only FLOPS can therefore create a misleading impression of performance. A data-processing pipeline may be valuable even when its floating-point rate is low.

Utilization and Efficiency

Utilization describes how much of an available resource is active. Processor utilization may be approximated as the proportion of time processors perform useful work rather than waiting. Hardware monitoring systems may also report memory-bandwidth utilization, accelerator utilization, network utilization, or storage utilization.

Efficiency compares achieved performance with a reference or limit. For parallel computing, parallel efficiency is commonly defined as

\[E_p = \frac{S_p}{p} = \frac{T_1}{pT_p},\]

where \(p\) is the number of processing elements, \(T_1\) is the serial execution time, \(T_p\) is the parallel execution time, and \(S_p\) is speedup. Efficiency is often expressed as a percentage. An efficiency of 0.75 means that the observed speedup is 75 percent of ideal linear speedup.

Utilization and efficiency are related but not identical. A processor can be highly occupied while executing unnecessary work, waiting on memory, or performing redundant communication. Effective performance requires useful work, not merely activity.

From Serial Execution to Parallel Speedup

Speedup measures the benefit obtained from parallel execution:

\[S_p = \frac{T_1}{T_p}.\]

If a job takes 100 seconds on one processor and 25 seconds on four processors, its speedup is 4 and its efficiency is 100 percent. If it takes 30 seconds, speedup is 3.33 and efficiency is approximately 83.3 percent.

Linear speedup, \(S_p=p\), is an ideal condition. Superlinear speedup, \(S_p>p\), can occasionally occur because the parallel version fits better in cache, uses a different search pattern, or benefits from other architectural effects. It should not automatically be interpreted as evidence of a superior parallel algorithm; measurement conditions and the serial baseline require examination.

Speedup is meaningful only when the compared executions solve equivalent problems and use comparable implementation quality. A poorly optimized serial baseline can make parallel speedup appear artificially large. Similarly, changing the algorithm, precision, stopping criterion, or output requirements invalidates a simple speedup comparison.

A useful decomposition of parallel execution time is

\[T_p = T_{work}+T_{communication}+T_{synchronization}+T_{I/O}+T_{overhead}.\]

These components may overlap, so the equation is conceptual rather than an exact accounting identity in every system. As processor count increases, the useful-work component may decline while communication, synchronization, scheduling, and memory contention become relatively more important.

Strong and Weak Scaling

Strong scaling asks: how does execution time change when the fixed problem is divided among more processors? An ideally strong-scalable program reduces time in proportion to processor count. Strong scaling is important when an organization must complete a fixed analysis or simulation within a deadline.

Weak scaling asks: how does execution time change when both the problem size and processor count increase proportionally, keeping work per processor approximately constant? An ideally weak-scalable program maintains nearly constant execution time. Weak scaling reflects situations in which additional resources are used to solve a larger problem rather than to complete the same problem faster.

These definitions must be reported explicitly. A graph showing execution time against processor count is not interpretable unless the workload is known. Strong scaling generally exposes serial fractions and fixed costs; weak scaling often exposes communication growth, global coordination, memory pressure, and network bottlenecks.

Suppose a model trains on 100 million records using 8 workers. Increasing to 32 workers while retaining the same records tests strong scaling. Increasing both workers and records by a factor of four tests weak scaling. The two experiments answer different operational questions and should not be substituted for one another.

Amdahl’s Law: The Limit of a Fixed Workload

Amdahl’s Law models the maximum speedup of a fixed-size problem when a fraction of execution is serial. If \(s\) is the serial fraction and \(p\) processors are used, the predicted speedup is

\[S(p)=\frac{1}{s+(1-s)/p}.\]

As \(p\) approaches infinity,

\[S_{max}=\frac{1}{s}.\]

Thus, if 10 percent of a workload is inherently serial, maximum speedup is 10, regardless of the number of processors. With 100 processors, the predicted speedup is approximately 9.17, not 100.

The serial fraction includes more than explicitly sequential code. It may include input/output, initialization, lock contention, sequential reductions, load imbalance, communication, and any overhead that grows into a limiting component. Therefore, the observed limit may be worse than the algorithm’s theoretical serial fraction suggests.

Amdahl’s Law is particularly useful for procurement and optimization decisions. If profiling shows that a small serial phase dominates at high processor counts, buying additional processors is unlikely to provide value until that phase is redesigned. However, Amdahl’s model assumes a fixed problem size and simplified overheads; it should be treated as a model for reasoning, not as a complete performance prediction.

Gustafson’s Law: Scaling the Problem with the Machine

Gustafson’s Law addresses a different situation: larger machines are used to solve larger problems within an acceptable time. If \(s\) is the fraction of execution time spent serially on a parallel machine with \(p\) processors, scaled speedup is often written as

\[S_G(p)=p-s(p-1).\]

For example, if the serial fraction is 0.1 and 64 processors are used, scaled speedup is

\[64-0.1(63)=57.7.\]

This result is substantially more optimistic than the fixed-size prediction because the parallel portion grows with the problem while the serial portion remains relatively bounded.

The two laws are not contradictory. Amdahl’s Law asks how quickly one fixed job can be completed; Gustafson’s Law asks how much larger a job can be completed in a similar elapsed time. Data science workflows frequently exhibit both perspectives. An organization may first seek a faster response for a fixed dataset and later use the same infrastructure to analyze higher-resolution data, longer time periods, or more model candidates.

Arithmetic Intensity and the Roofline Model

Arithmetic intensity is the ratio of arithmetic operations to data moved between a relevant memory level and the processor:

\[I = \frac{\text{operations}}{\text{bytes transferred}}.\]

The appropriate memory level matters. An intensity calculated relative to main memory differs from one calculated relative to cache or accelerator memory. Arithmetic intensity is not simply the size of the input divided by the number of operations; it concerns data movement during execution.

The roofline model relates attainable performance to arithmetic intensity. If peak computational performance is \(P_{peak}\) and sustainable memory bandwidth is \(B\), the performance bound is

\[P \leq \min(P_{peak}, B \times I).\]

The intersection occurs at a critical intensity

\[I^* = \frac{P_{peak}}{B}.\]

Below this intensity, the workload is generally memory-bound: increasing arithmetic capability alone is unlikely to help. Above it, the workload is generally compute-bound: optimization should focus on instruction efficiency, vectorization, parallel execution, and utilization of arithmetic units.

Consider a processor with peak performance of 2 TFLOPS and sustainable memory bandwidth of 200 GB/s. The ridge point is 10 FLOPs per byte. An application with intensity 2 FLOPs/byte has a roofline performance bound of 400 GFLOPS. An application with intensity 20 FLOPs/byte is bounded by the 2 TFLOPS compute roof, although real performance may be lower because of instruction overhead or imperfect parallelism.

The roofline model is deliberately simple. It does not explicitly represent latency, cache effects, communication, synchronization, irregular access, or multiple memory levels. Nevertheless, it provides a powerful first diagnosis: should optimization reduce data movement or increase computation efficiency?

Benchmarking as an Experimental Discipline

Benchmarking is not the act of timing a program once. It is an experiment that must control conditions, define the workload, collect repeated observations, and report uncertainty. A credible benchmark begins with a precise question. Examples include: Does a GPU reduce inference time for a fixed batch? How does a distributed join scale from 4 to 64 nodes? Which storage configuration provides the best throughput at an acceptable cost?

The benchmark specification should identify the input dataset, data distribution, algorithm and software version, compiler flags, hardware, processor count, precision, initialization procedure, and whether input/output time is included. Warm-up runs may be necessary to populate caches, compile kernels, or establish steady-state behavior, but they should be distinguished from cold-start measurements.

Repeated trials are essential because execution time varies with operating-system activity, network contention, frequency scaling, thermal conditions, background jobs, garbage collection, and data-dependent behavior. The arithmetic mean is useful when extreme values are not dominant, while the median is often more robust for skewed latency measurements. Percentiles such as the 95th or 99th are important for service-level objectives. A result should include dispersion, such as standard deviation, interquartile range, or a confidence interval, rather than a single attractive number.

Outliers should not be removed merely because they weaken the conclusion. A documented rule may exclude a failed run caused by an identified external fault, but unexplained outliers are evidence requiring investigation. Statistical significance and practical significance should also be distinguished. A tiny improvement may be statistically detectable but operationally irrelevant if it does not offset additional hardware or management cost.

Benchmark comparisons should use paired conditions where possible. Running alternatives under the same input, node allocation, and environmental conditions reduces noise. Randomized trial order can reduce bias from thermal or system state. Profilers and hardware counters should complement timing data by revealing cache misses, branch mispredictions, memory bandwidth, synchronization, and accelerator occupancy.

A strong report states the measurement boundary. For example, “kernel time” excludes data transfer and initialization, whereas “end-to-end latency” includes them. Both may be valid, but they answer different questions. Reproducibility requires recording configuration, scripts, versions, environment variables, and data-generation procedures.

A Practical Performance-Analysis Workflow

Performance analysis normally proceeds from an operational objective to a diagnosis and then to controlled optimization. First, define the service requirement: completion time, sustained throughput, tail latency, energy, cost, or a combination. Second, establish a correct baseline and verify that outputs are equivalent. Third, measure end-to-end behavior before focusing on individual kernels. Fourth, profile to identify the dominant resource or phase. Fifth, formulate a hypothesis, such as insufficient memory locality or excessive synchronization. Sixth, change one major factor at a time and repeat the benchmark. Finally, test the optimized version across representative workloads rather than a single favorable case.

This workflow prevents a common error: optimizing the most visible code instead of the most consequential bottleneck. If a program spends 70 percent of its time reading data, a 2× improvement to computation that occupies 20 percent of runtime produces only a modest end-to-end benefit. Amdahl-style reasoning helps estimate the maximum value of each proposed optimization.

Illustrative Case Study: Scaling a Distributed Feature-Engineering Pipeline

Background

The following is an illustrative case involving a data-science team that generates features from 2 TB of event records for a fraud-detection model. The pipeline parses records, joins them with reference data, computes rolling statistics, and writes partitioned files for model training.

Situation and Problem

On four worker nodes, the pipeline completes in 58 minutes. Management proposes expanding to 32 nodes and expects an eightfold reduction to approximately 7.25 minutes. The team must determine whether that expectation is realistic and whether the additional nodes are economically justified.

Analysis

A strong-scaling experiment keeps the 2 TB dataset fixed. The following illustrative measurements are obtained after three warm-up runs and ten recorded trials:

• 4 nodes: 58.0 minutes
• 8 nodes: 31.5 minutes
• 16 nodes: 18.4 minutes
• 32 nodes: 13.9 minutes

The speedup at 32 nodes is \(58/13.9\), approximately 4.17, and parallel efficiency relative to four nodes is \(4.17/8\), approximately 52 percent. The result is far from ideal, but adding nodes still reduces completion time.

Profiling indicates that parsing and feature arithmetic occupy less time as nodes increase, while the global join and output commit become increasingly dominant. Hardware counters show moderate processor utilization but high network traffic and synchronization delay. The arithmetic intensity of the feature calculations is low, suggesting that the computation is partly memory- and communication-bound rather than limited by floating-point capacity.

A weak-scaling experiment increases both data and nodes: 0.5 TB on one node, 2 TB on four nodes, and 16 TB on 32 nodes. Runtime rises from 15 minutes to 22 minutes, indicating that the system does not maintain constant time as the workload grows. The likely causes are metadata coordination, skewed key distributions, and a reference table that is repeatedly transferred rather than efficiently partitioned or cached.

Approach and Strategy

The team first separates kernel timing from end-to-end timing and confirms that output correctness is unchanged. It then applies three interventions: partitioning the reference data by join key, caching read-only reference data on each worker, and replacing a centralized output commit with parallel object-store writes followed by a lightweight manifest operation. Data skew is measured explicitly, and unusually large partitions are divided into subpartitions.

After these changes, the 32-node runtime falls to 9.8 minutes, but the team does not claim an eightfold speedup without comparing the revised implementation with the original baseline under identical conditions. The cost analysis shows that 16 nodes provide a better cost-per-completed-pipeline measure for routine daily jobs, while 32 nodes are justified only when a reporting deadline requires the lower latency.

Outcome and Learning Points

The case demonstrates that processor count alone did not determine performance. Strong scaling exposed fixed coordination and output costs; weak scaling exposed data movement and skew. The roofline perspective suggested that adding floating-point capacity would not solve the dominant problem. Benchmark repetition and clear timing boundaries prevented the team from confusing kernel improvement with end-to-end improvement. Finally, the best engineering decision depended not only on speedup but also on cost, deadline, workload variability, and operational risk.

Common Errors, Risks, and Limitations

A frequent mistake is comparing peak hardware specifications rather than achieved application performance. Another is reporting speedup without specifying the serial baseline, workload, or inclusion of input/output. Results may also be distorted by comparing different data sizes, using an unoptimized serial program, or allowing one implementation to use a more favorable algorithm.

Scaling graphs can hide important behavior. Average runtime may conceal poor tail latency, and aggregate throughput may conceal severe imbalance among workers. A benchmark based on a synthetic dataset may not represent production distributions, especially for skewed joins, sparse matrices, or irregular graphs. Reproducibility can be weakened by undocumented software versions, dynamic cloud allocation, or shared infrastructure.

Performance optimization also creates risks. Aggressive compiler transformations may affect numerical reproducibility. Reduced precision can improve throughput but alter model accuracy or scientific conclusions. Caching sensitive datasets on local devices may increase privacy exposure. Performance logs can contain identifiers or workload information that require governance and access control.

Professional and Organizational Implications

Performance metrics support architectural and financial decisions. A manager evaluating a cluster should consider performance per node, performance per unit cost, energy per completed job, utilization over time, and queue delay—not merely peak FLOPS. A platform team may choose fewer powerful nodes for latency-sensitive workloads but more economical nodes for highly parallel batch processing.

Engineering teams should establish performance budgets, regression tests, and service-level objectives. A change that improves one benchmark but increases memory consumption, cloud expenditure, or operational complexity may not be an overall improvement. Governance should require documented benchmark methods for procurement, capacity planning, and claims made to stakeholders.

Emerging Directions

Modern performance analysis increasingly spans CPUs, GPUs, specialized accelerators, high-bandwidth memory, distributed storage, and high-speed interconnects. The traditional single roofline may be extended to account for multiple memory levels, communication roofs, or heterogeneous devices. Automated performance tuning systems use profiling data and machine learning to explore block sizes, placement, kernel variants, and scheduling policies.

Energy and carbon efficiency are also becoming first-class metrics. A lower execution time is not necessarily environmentally preferable if it requires disproportionately more power. Similarly, reproducible benchmarking is gaining importance as cloud systems, container orchestration, and shared accelerators introduce variable environments. The fundamental principles remain unchanged: define the question, measure the relevant boundary, quantify uncertainty, and connect results to an operational decision.

Best Practices for Reliable Performance Decisions

A sound practice is to:

• define latency, throughput, and completion requirements before measuring;
• use a correct and appropriately optimized baseline;
• distinguish strong-scaling from weak-scaling experiments;
• report achieved performance rather than relying on peak specifications;
• use profiling to connect metrics with bottlenecks;
• repeat trials and report variability and relevant percentiles;
• preserve workload, software, hardware, and environment metadata;
• evaluate performance, cost, energy, accuracy, and maintainability together;
• verify numerical and functional equivalence after optimization; and
• treat benchmark results as evidence for a decision, not as universal properties of a machine.

Chapter Summary

Performance measurement converts the vague idea of a “fast” system into observable quantities such as execution time, latency, throughput, FLOPS, utilization, and efficiency. Speedup and parallel efficiency quantify the benefits of additional processors, while strong scaling evaluates a fixed workload and weak scaling evaluates growth in workload with resources. Amdahl’s Law explains the limits of accelerating a fixed problem; Gustafson’s Law explains why larger problems may justify larger machines. Arithmetic intensity and the roofline model help distinguish memory-bound from compute-bound behavior. Reliable benchmarking requires defined measurement boundaries, repeated trials, controlled conditions, variability reporting, and reproducibility. In professional HPC practice, the best solution balances performance with cost, energy, correctness, security, and operational requirements.

Key Terms

Execution time: Elapsed time required to complete a workload.

Latency: Time required to complete one operation or respond to one request.

Throughput: Amount of work completed per unit time.

FLOPS: Floating-point operations performed per second.

Peak performance: Theoretical maximum computational rate of a system.

Achieved performance: Measured computational rate delivered by an application.

Utilization: Proportion of an available resource engaged in activity.

Efficiency: Ratio of achieved performance to an ideal, reference, or maximum value.

Speedup: Ratio of baseline execution time to parallel execution time.

Parallel efficiency: Speedup divided by the number of processing elements.

Strong scaling: Scaling a fixed problem over an increasing number of processors.

Weak scaling: Increasing problem size with processor count while holding work per processor approximately constant.

Serial fraction: Portion of execution that cannot benefit from parallel processing.

Amdahl’s Law: Model of speedup limits for a fixed-size workload.

Gustafson’s Law: Model of scaled speedup when workload grows with processor count.

Arithmetic intensity: Number of arithmetic operations per byte of data transferred.

Roofline model: Performance-bound model based on peak computation and memory bandwidth.

Memory-bound: Condition in which data movement limits performance.

Compute-bound: Condition in which arithmetic capability limits performance.

Benchmark: Controlled measurement used to evaluate system or application performance.

Review Questions

Short Answer Questions

1. Distinguish latency from throughput using an HPC or data-science example.

2. Why can achieved FLOPS be much lower than advertised peak FLOPS?

3. Define strong scaling and weak scaling.

4. What does arithmetic intensity measure?

5. Why should benchmark results include variability rather than only an average time?

Long Answer Questions

1. Explain speedup and parallel efficiency, including the assumptions required for a valid comparison.

2. Derive and interpret Amdahl’s Law for a workload with a non-zero serial fraction.

3. Compare Amdahl’s Law and Gustafson’s Law and explain the operational question addressed by each.

4. Explain the roofline model and show how it guides optimization of memory-bound and compute-bound applications.

5. Describe a complete methodology for designing and reporting a statistically sound HPC benchmark.

Analytical / Application Questions

1. A program takes 120 seconds on one processor and 20 seconds on eight processors. Calculate speedup and parallel efficiency. Interpret the result.

2. A system has peak performance of 4 TFLOPS and sustainable bandwidth of 400 GB/s. Determine the roofline ridge point and the performance bound for an application with arithmetic intensity of 3 FLOPs/byte.

3. A distributed pipeline shows excellent kernel speedup but nearly constant end-to-end time as nodes increase. Identify at least three likely causes and propose a measurement strategy to distinguish them.

22. Multiple Choice Questions with Answers

Question 1

A workload takes 80 seconds on one processor and 25 seconds on four processors. What are its speedup and parallel efficiency?

A. Speedup 3.2; efficiency 80%

B. Speedup 3.2; efficiency 32%

C. Speedup 4; efficiency 80%

D. Speedup 2.5; efficiency 62.5%

Correct Answer: A. Speedup 3.2; efficiency 80%

Question 2

Which experiment is a strong-scaling experiment?

A. Increasing both data size and processors by the same factor

B. Keeping the dataset fixed while increasing the number of processors

C. Keeping processor count fixed while increasing data size

D. Changing the algorithm while retaining processor count

Correct Answer: B. Keeping the dataset fixed while increasing the number of processors

Question 3

If the serial fraction of a workload is 0.2, what is its theoretical maximum speedup under Amdahl’s Law?

A. 2

B. 4

C. 5

D. 20

Correct Answer: C. 5

Question 4

What does a low arithmetic intensity usually suggest?

A. The application is likely limited by data movement or memory bandwidth

B. The application must achieve peak FLOPS

C. The application has no parallelism

D. The application is necessarily limited by storage capacity

Correct Answer: A. The application is likely limited by data movement or memory bandwidth

Question 5

A processor has a peak of 2 TFLOPS and memory bandwidth of 200 GB/s. What is the roofline ridge point?

A. 0.1 FLOPs/byte

B. 2 FLOPs/byte

C. 10 FLOPs/byte

D. 400 FLOPs/byte

Correct Answer: C. 10 FLOPs/byte

Question 6

Which metric is most appropriate for describing the number of records processed per second?

A. Latency

B. Throughput

C. Serial fraction

D. Arithmetic intensity

Correct Answer: B. Throughput

Question 7

Why is reporting only a single benchmark run generally inadequate?

A. A single run cannot calculate FLOPS

B. Execution time can vary because of system and workload conditions

C. Speedup is defined only for ten runs

D. Peak performance changes after every run

Correct Answer: B. Execution time can vary because of system and workload conditions

Question 8

Under Gustafson’s perspective, adding processors is especially justified when it enables an organization to:

A. Solve a larger problem in approximately the same time

B. Eliminate all serial execution

C. Guarantee linear speedup for a fixed problem

D. Avoid measuring communication overhead

Correct Answer: A. Solve a larger problem in approximately the same time

Question 9

A benchmark reports kernel time but excludes data transfer and initialization. What is the main implication?

A. It is automatically an end-to-end latency measurement

B. It measures only a defined component and may not represent user-perceived runtime

C. It cannot be repeated statistically

D. It must be measuring throughput

Correct Answer: B. It measures only a defined component and may not represent user-perceived runtime

Question 10

A workload’s processor utilization is high, but its performance is poor and memory bandwidth is saturated. Which interpretation is most appropriate?

A. High utilization proves the program is compute-bound

B. The processors may be active while waiting on or servicing data movement

C. The program has achieved linear speedup

D. The workload has no communication overhead

Correct Answer: B. The processors may be active while waiting on or servicing data movement