Experimental Design, Data Analytics and Technical Reporting

Introduction and Context

Thermal and manufacturing engineering laboratories convert physical principles into evidence. A laboratory investigation is not merely a sequence of measurements; it is a controlled argument about how a system behaves. The quality of that argument depends on the question posed, the experimental design, the measurement system, the treatment of uncertainty, the analysis of data, and the clarity with which results are reported.

In thermal engineering, experiments may involve heat transfer, fluid flow, refrigeration, combustion, or energy conversion. In manufacturing, they may concern machining forces, surface roughness, dimensional accuracy, tool wear, additive manufacturing parameters, or material processing. Although the equipment differs, the intellectual process is similar: formulate a hypothesis, identify variables, obtain representative data, analyze variation, compare observations with theory, and communicate conclusions that are proportional to the evidence.

A technically impressive graph cannot compensate for uncontrolled variables or poorly documented measurements. Conversely, a simple experiment can produce valuable engineering knowledge when its design and limitations are explicit.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Formulate testable hypotheses and identify dependent, independent, controlled, and nuisance variables.
2. Design experiments using control, randomization, replication, blocking, and appropriate sampling.
3. Distinguish accuracy, precision, resolution, sensitivity, repeatability, and reproducibility.
4. Organize laboratory data and calculate descriptive statistics and uncertainty measures.
5. Select suitable tables, plots, regression models, and statistical tests for engineering data.
6. Validate theoretical models using residuals, error measures, and uncertainty intervals.
7. Diagnose systematic, random, gross, and procedural errors.
8. Maintain auditable laboratory records and prepare reproducible technical reports.
9. Recognize ethical, safety, data-governance, and professional responsibilities in experimental work.

1. From Engineering Question to Testable Hypothesis

A useful experiment begins with a precise question. “Investigate heat transfer” is too broad, whereas “Determine whether increasing air velocity increases the convective heat-transfer coefficient of a heated cylinder within the tested range” is testable.

A hypothesis is a provisional statement that predicts an observable relationship. For example:

Increasing cutting speed, while feed rate and depth of cut remain constant, will increase tool temperature.

The independent variable is deliberately changed by the investigator. The dependent variable is measured as the response. Controlled variables are held constant as far as practical. Nuisance variables influence the response but are not the main focus; ambient temperature, tool condition, electrical supply fluctuations, and operator technique are common examples.

A null hypothesis, H₀, generally states that there is no specified effect or difference. An alternative hypothesis, H₁, states the effect that the investigation is designed to detect. Statistical testing does not prove a hypothesis absolutely. It assesses whether the observed evidence is sufficiently inconsistent with a chosen null model.

A strong hypothesis has four characteristics:

• it identifies the variables;
• it predicts direction or difference where justified;
• it is measurable using available equipment;
• it specifies the operating range or conditions.

Operational definitions

An engineering term must be translated into a measurement procedure. For example, “surface quality” may be defined as arithmetic mean roughness, R_a, measured in micrometres using a specified instrument, cutoff length, and number of traces. “Steady state” should be defined by a criterion such as a temperature change below a stated threshold over a specified time interval. Operational definitions prevent different investigators from assigning different meanings to the same variable.

2. Experimental Design Principles

2.1 Control and comparison

A control condition provides a baseline against which a treatment or changed condition can be judged. In a heat-transfer experiment, the control may be natural convection at a specified surface temperature; in machining, it may be a standard cutting speed. A control need not be an untreated specimen, but it must be clearly defined and measured under comparable conditions.

Comparison is strongest when only the factor of interest changes. If cutting speed, feed rate, workpiece material, and tool geometry all change simultaneously, the resulting effect cannot be attributed confidently to any single factor.

2.2 Replication and repeated measurement

Replication means independently repeating an experimental condition. It estimates variation in the experimental process. Repeated readings taken from the same uninterrupted run may reveal instrument noise but do not fully represent run-to-run variation. The distinction is important:

• Repeatability concerns agreement under the same conditions, operator, instrument, and short time interval.
• Reproducibility concerns agreement when conditions such as operator, instrument, laboratory, or day change.

Replication should be planned before data collection. Three repeated readings are not automatically sufficient; the number depends on expected variability, available resources, and the precision required.

2.3 Randomization

Randomization prevents time-related or unrecognized trends from being systematically associated with a treatment. If five cutting speeds are always tested from lowest to highest, tool wear and machine heating may be confounded with cutting speed. A randomized order, or a carefully justified order, reduces this risk. Randomization does not mean careless operation: safety constraints and warm-up requirements must still be respected.

2.4 Blocking

Blocking groups experimental runs that share a known source of variation. For example, tests performed on different days can be treated as blocks, with all cutting speeds represented within each day if possible. Blocking removes part of the background variation from the treatment comparison.

2.5 Factorial thinking

When two or more factors may interact, changing one factor at a time can be inefficient and misleading. A factorial design deliberately combines factor levels. For factors A and B, a two-level full factorial design includes the combinations A−B−, A+B−, A−B+, and A+B+.

An interaction occurs when the effect of one factor depends on the level of another. In machining, the effect of cutting speed on surface roughness may differ at low and high feed rates. Interaction is often visible when lines on an interaction plot are non-parallel.

3. Sampling and Measurement Planning

The population is the full set of possible units about which conclusions are intended—for example, all parts produced by a process under defined conditions. A sample is the subset actually measured. The sampling frame is the practical list or process from which the sample is obtained.

Common sampling approaches include:

Sampling must also address time. Thermal systems drift, tools wear, and environmental conditions change. A sample taken only at the beginning of a production run may not represent the complete process. Time-stamped observations and planned intervals make such effects visible.

Measurement-system planning

Before taking measurements, specify the measurand, instrument, calibration status, range, resolution, mounting method, sampling frequency, and environmental conditions. The instrument should be appropriate for the expected magnitude and rate of change. A thermocouple with a coarse display may be unsuitable for a small temperature difference, even if its nominal range is adequate.

Calibration establishes the relationship between instrument indication and reference values. Calibration does not guarantee perfect measurements; it provides evidence of instrument behaviour and may reveal bias. Calibration records should include the instrument identity, date, standard used, results, and next due date.

4. Data Integrity and Laboratory Records

A laboratory notebook or electronic record is a primary technical document. It should allow another competent person to reconstruct what was done. Record:

• date, time, operator, equipment identification, and software version;
• objective, hypothesis, and experimental plan;
• specimen identity, material, dimensions, and preparation;
• instrument range, resolution, calibration status, and settings;
• environmental and operating conditions;
• raw observations, including unusual readings;
• deviations from the planned procedure;
• calculations, code, file names, and data-processing decisions.

Raw data should not be overwritten by corrected or processed values. Corrections should be traceable, dated, justified, and linked to the original entry. In electronic work, version control, read-only raw-data storage, file naming conventions, and metadata improve reproducibility.

A suitable filename might include project, date, run, and revision: HTC_2025-03-14_Run03_v02.csv. Units must be recorded in the data table, not inferred later.

5. Descriptive Data Analysis

Suppose n observations of a quantity x are represented by x₁, x₂, …, xₙ. The arithmetic mean is:

Here x̄ has the same units as x. The mean is appropriate for measurements whose deviations are reasonably balanced and for which extreme values are not dominant.

The sample variance is:

The sample standard deviation is s = √s². The denominator n − 1 is used when the sample standard deviation estimates the variability of a wider population. The standard deviation describes scatter among individual observations; it is not the uncertainty of the mean.

The standard error of the mean is:

This expression assumes observations are independent and identically distributed, or that the dependence has been appropriately addressed. Increasing n reduces the standard error only as the square root of n, so four times as many independent observations are needed to halve it.

For a roughly normal distribution, an approximate 95% confidence interval for the population mean is:

where t₀.₉₇₅, n−1 is the relevant Student’s t critical value with n − 1 degrees of freedom. This interval estimates a range for the population mean under the stated assumptions; it is not a guarantee that 95% of individual measurements lie inside the interval.

Worked example: repeated temperature readings

Five steady-state readings are 81.2, 80.8, 81.5, 81.0, and 80.5 °C. Their mean is:

x̄ = (81.2 + 80.8 + 81.5 + 81.0 + 80.5)/5 = 81.0 °C

The deviations from the mean are 0.2, −0.2, 0.5, 0.0, and −0.5 °C. The sum of squared deviations is 0.58 °C², so:

s² = 0.58/(5−1) = 0.145 °C²

and s = 0.381 °C. The standard error is SE = 0.381/√5 = 0.171 °C. The reported result should distinguish the scatter of readings from the uncertainty associated with estimating the mean.

Graphical analysis

Graphs should expose structure, not merely decorate a report. Recommended choices include:

• scatter plots for relationships between two quantitative variables;
• line plots for time-dependent measurements;
• dot plots or histograms for distributions;
• box plots for comparing groups;
• residual plots for model diagnosis;
• interaction plots for factorial experiments.

Axes require quantities, units, sensible scales, and readable tick marks. Error bars must be identified: they may represent standard deviation, standard error, confidence intervals, or instrument uncertainty, and these meanings are not interchangeable.

6. Uncertainty and Error Diagnosis

Measurement error is the difference between a measured value and the unknown true value. It is commonly classified as:

• Random error: unpredictable variation that produces scatter.
• Systematic error: consistent bias, such as a zero offset or heat loss omitted from a model.
• Gross error: mistakes such as misreading a scale, entering a value incorrectly, or using the wrong specimen.
• Drift: gradual change in instrument or system response.

Repeating measurements helps quantify random variation but does not remove systematic bias. Calibration, comparison with a reference, procedural checks, and physical modelling are needed to identify bias.

For a result y that depends on measured inputs x₁, x₂, …, xₖ, a first-order uncertainty propagation estimate for independent inputs is:

Here uⱼ is the standard uncertainty associated with xⱼ, and derivatives are evaluated at the measured values. This approximation assumes small uncertainties, a sufficiently smooth model, and negligible covariance between inputs. If inputs are correlated, covariance terms must be included.

For a simple heat-transfer calculation, the convective heat-transfer coefficient is often estimated by:

where h is in W·m⁻²·K⁻¹, Q̇ is convective heat-transfer rate in W, A is area in m², Tₛ is surface temperature, and T∞ is free-stream temperature. The expression assumes that Q̇ represents the convective heat transfer through the stated area and that radiation, conduction through supports, and other losses have either been removed or included in the energy balance. If these assumptions are not satisfied, a precise numerical value of h may still be physically misleading.

A useful diagnostic is an error budget. List each important input, its estimated uncertainty, sensitivity, and contribution to the final result. This identifies whether better temperature measurement, flow measurement, geometry measurement, or experimental control would most improve the result.

7. Model Fitting and Validation

A theoretical model expresses an expected relationship. For example, a linear calibration model may be written as:

where y is the response, x is the predictor, β₀ is the intercept, β₁ is the slope, and ε represents unexplained variation. Least-squares fitting estimates β₀ and β₁ by minimizing the sum of squared residuals:

The residual for observation i is eᵢ = yᵢ − ŷᵢ. A fitted relationship is not validated merely because its coefficient of determination is high. Residual plots should be checked for curvature, changing spread, outliers, and time patterns.

The coefficient of determination is:

where SST = ∑ᵢ₌₁ⁿ(yᵢ − ȳ)². R² is the fraction of sample variation explained by the fitted model in the usual least-squares setting. It does not establish causation, physical correctness, or applicability outside the tested range.

Model validation should include:

1. Dimensional consistency: predicted and measured quantities have compatible units.
2. Assumption checking: steady state, one-dimensional behaviour, negligible losses, independence, or normality are examined rather than silently assumed.
3. Residual analysis: residuals are plotted against predictors, time, and fitted values.
4. Independent comparison: where possible, validation data are not the same data used to fit parameters.
5. Error comparison: differences are assessed relative to measurement uncertainty and engineering requirements.

A percentage error is often written as:

This measure is unsuitable when the reference value is zero or very close to zero. Also, percentage error alone does not indicate whether a discrepancy is statistically or practically significant.

8. Technical Reporting

A technical report is a reproducible record and an engineering argument. Its structure should reflect the investigation rather than imitate a laboratory manual.

Recommended structure

Title and identification: State the experiment, author, date, course, and equipment where appropriate.

Abstract: Briefly state purpose, method, principal quantitative result, and conclusion. It should stand alone.

Introduction: Establish context, engineering significance, theory, objectives, and hypotheses. Cite authoritative sources where required.

Methods: Describe apparatus, materials, variables, settings, calibration, procedure, sampling, and data processing sufficiently for replication. Do not hide important deviations.

Results: Present processed results with units, uncertainty information, tables, graphs, and relevant statistical outputs. Avoid interpreting every result in this section if a separate discussion is used.

Discussion: Explain trends, compare results with theory, assess assumptions, identify error sources, and distinguish evidence from speculation.

Conclusion: Answer the original objectives directly. Do not introduce new results.

Recommendations: Suggest justified changes to apparatus, procedure, sampling, or future experiments.

References and appendices: Include sources, raw data, detailed calculations, code, calibration documents, and supplementary plots as appropriate.

Tables should have numbered captions and column units. Figures should be cited in the text and should remain understandable without excessive reference to surrounding prose. Significant figures should reflect measurement capability. Reporting 25.43782 °C from an instrument readable only to 0.1 °C creates false precision.

Reproducible analysis

A reproducible workflow separates raw data, processed data, scripts, figures, and report outputs. Analysis code should identify input files, transformations, statistical methods, software versions, and random seeds when random procedures are used. Manual spreadsheet operations should be minimized or documented. A second person should be able to rerun the analysis and obtain the reported values.

9. Illustrative Case: Heat-Transfer Experiment

Background

A laboratory team investigates forced convection from a heated cylinder. The engineering question is whether increasing air velocity increases the estimated heat-transfer coefficient.

Problem and design

The team defines air velocity as the independent variable and estimates h as the response. Surface temperature, air temperature, electrical power, cylinder geometry, and thermocouple locations are controlled. Three independent runs are planned at each of four air velocities. Run order is randomized within each block of tests, and the apparatus is allowed to reach a defined steady-state criterion before readings are recorded.

The team recognizes that electrical input is not necessarily equal to convective heat transfer. Power lost by radiation, conduction through supports, and wiring must be estimated or measured. Thus the energy balance is recorded as:

Q̇_electrical = Q̇_convection + Q̇_radiation + Q̇_conduction + Q̇_storage

At steady state, Q̇_storage ≈ 0, but the other loss terms may remain important.

Analysis

For each run, the team records voltage, current, surface temperature, ambient temperature, air velocity, and time. It calculates electrical power as:

where P is power in watts, V is voltage in volts, and I is current in amperes, assuming direct-current conditions or an appropriately measured real power for alternating-current equipment. The team then subtracts estimated non-convective losses before calculating h.

The results are plotted as h against air velocity with uncertainty bars. Residuals from a fitted trend are inspected. The team notices that the first run at each velocity has higher temperatures because the apparatus was not fully stabilized. Those observations are flagged in the record, not silently deleted. A repeat experiment confirms that stabilization time, rather than air velocity, caused the apparent anomaly.

Outcome

After applying the defined steady-state criterion and accounting for heat losses, h increases with velocity over the tested range. The conclusion is limited to the cylinder geometry, air-property range, velocity range, and operating conditions studied. The team does not claim universal validity for all forced-convection systems.

Learning points

The case demonstrates that experimental design precedes statistical analysis. Replication reveals run-to-run variation; time records identify thermal drift; an energy balance prevents an inappropriate interpretation of electrical power; and uncertainty analysis shows whether the observed trend is larger than measurement variation.

10. Limitations, Ethics, Safety, and Contemporary Practice

No experiment is free from limitations. Small samples may have low statistical power, laboratory conditions may not represent industrial environments, and model assumptions may fail at parameter extremes. Extrapolation beyond the measured range should be clearly labelled as prediction rather than observation.

Ethical laboratory practice requires honest reporting of raw data, uncertainty, failed runs, exclusions, and deviations. Data must not be fabricated, selectively removed to improve agreement, or presented with unjustified precision. Authorship and contribution should be transparent. Where student or industrial data contain confidential information, access controls and anonymization may be necessary.

Safety is part of experimental validity. A run that risks burns, rotating-equipment entanglement, pressure release, electrical shock, or hazardous fumes is not acceptable merely because it may produce useful data. Risk assessment, guarding, personal protective equipment, emergency procedures, and equipment interlocks must be addressed before operation.

Digital sensors, automated acquisition, machine vision, cloud storage, and artificial intelligence can improve data collection and pattern recognition. However, automation can also conceal sensor saturation, synchronization errors, software bugs, or inappropriate model assumptions. AI-generated analysis must be independently checked, documented, and never treated as an authority. Data provenance, cybersecurity, privacy, and version control are increasingly important in connected laboratories.

Best-Practice Checklist

Before the experiment, define the question, variables, hypothesis, sampling plan, uncertainty target, safety controls, and calibration requirements. During the experiment, preserve raw data, record time and conditions, follow the randomized or blocked plan, and document deviations. After the experiment, inspect data before deleting anything, plot raw observations, quantify uncertainty, check model assumptions, compare with theory, and report limitations. Before submission, verify units, significant figures, calculations, figure references, citations, and reproducibility of the analysis.

Chapter Summary

Experimental engineering is a disciplined process linking a question to defensible evidence. A testable hypothesis identifies measurable variables and predicts a relationship. Experimental validity is strengthened through controls, randomization, replication, blocking, appropriate sampling, and careful definition of the measurand. Measurement quality depends on calibration and on distinguishing accuracy, precision, repeatability, reproducibility, resolution, and uncertainty.

Descriptive statistics summarize data, while graphs and residuals reveal patterns that numerical summaries may hide. Standard deviation describes individual scatter, whereas the standard error describes uncertainty in an estimated mean under suitable assumptions. Uncertainty propagation and error budgets identify which measurements most influence a result. Model validation requires dimensional checks, assumption assessment, residual analysis, independent comparison where possible, and conclusions limited to the tested domain.

A professional technical report is an auditable and reproducible engineering argument. It preserves raw data, explains methods, reports results with units and uncertainty, discusses discrepancies honestly, and states conclusions proportional to evidence. Safety, ethical data handling, software transparency, and responsible use of automation are integral—not optional—to laboratory competence.

Key Terms

Hypothesis: A testable prediction about an observable relationship.

Null hypothesis: A statistical statement representing no specified effect or difference.

Independent variable: A factor deliberately varied by the investigator.

Dependent variable: The measured response affected by experimental conditions.

Controlled variable: A factor held constant to isolate the effect of interest.

Nuisance variable: An uncontrolled factor that may influence the response.

Replication: Independent repetition of an experimental condition.

Randomization: Assignment of run order or treatments by chance to reduce systematic bias.

Blocking: Grouping observations that share a known source of variation.

Accuracy: Closeness of a measurement to a reference or true value.

Precision: Degree of agreement among repeated measurements.

Repeatability: Agreement under substantially identical measurement conditions.

Reproducibility: Agreement when important measurement conditions change.

Calibration: Comparison of an instrument with a reference standard.

Standard deviation: A measure of dispersion among observations.

Standard error: An estimate of the standard deviation of a sampling statistic, commonly the mean.

Residual: Difference between an observed value and its model-predicted value.

Uncertainty: Quantified lack of complete knowledge about a measured result.

Confounding: A situation in which effects of factors cannot be separated reliably.

Reproducibility: The ability of another investigator to repeat the analysis and obtain consistent results.

Review Questions

Short Answer Questions

1. Distinguish an independent variable from a dependent variable using a thermal laboratory example.
2. Why does replication not necessarily remove systematic error?
3. State two differences between standard deviation and standard error.
4. What information should be recorded about a measurement instrument before data collection?
5. Why should a technical report state the range over which a model was tested?

Long Answer Questions

1. Explain how control, randomization, replication, and blocking can be combined in a machining experiment.
2. Describe a complete procedure for validating a theoretical heat-transfer model against laboratory data.
3. Discuss the causes and diagnosis of random, systematic, gross, and drift errors.
4. Explain how a reproducible data-analysis workflow should be organized from raw data to final report.
5. Discuss the ethical, safety, and professional responsibilities associated with laboratory data reporting.

Analytical / Application Questions

1. A student changes cutting speed from 20 to 40 m/min while feed rate, depth of cut, tool condition, and workpiece material also change. Identify the confounding problem and propose a revised design.
2. Five readings of a dimension are 12.1, 12.3, 12.2, 12.4, and 12.0 mm. Calculate the mean and sample standard deviation, then explain whether the result demonstrates accuracy.
3. An experimental heat-transfer coefficient agrees with theory within 4%, but the uncertainty of the experimental result is ±8%. Evaluate whether the agreement is meaningful and identify further evidence that should be examined.

Multiple Choice Questions with Answers

Question 1

Which variable is deliberately changed by the investigator in an experiment?

A. Dependent variable

B. Independent variable

C. Nuisance variable

D. Residual variable

Correct Answer: B. Independent variable

Question 2

What is the primary purpose of randomizing experimental run order?

A. To increase instrument resolution

B. To eliminate all random error

C. To reduce association between treatment and time-related effects

D. To guarantee a normal distribution

Correct Answer: C. To reduce association between treatment and time-related effects

Question 3

For n independent observations, the standard error of the sample mean is:

A. s × √n

B. s/n

C. s²/√n

D. s/√n

Correct Answer: D. s/√n

Question 4

Which practice is most effective for detecting a zero-offset bias?

A. Taking more readings without checking the instrument

B. Comparing the instrument with a suitable reference standard

C. Rounding all readings to fewer digits

D. Randomizing the report order

Correct Answer: B. Comparing the instrument with a suitable reference standard

Question 5

A residual is defined as:

A. The fitted value minus the sample mean

B. The observed value minus the model-predicted value

C. The standard deviation divided by sample size

D. The difference between two independent variables

Correct Answer: B. The observed value minus the model-predicted value

Question 6

Which statement about R² is correct?

A. A high R² proves that the model is physically correct

B. R² is always an estimate of measurement uncertainty

C. R² describes explained sample variation in the fitted model but does not prove causation

D. R² can be calculated only for nonlinear models

Correct Answer: C. R² describes explained sample variation in the fitted model but does not prove causation

Question 7

Five temperature readings are 81.2, 80.8, 81.5, 81.0, and 80.5 °C. What is their arithmetic mean?

A. 80.5 °C

B. 80.8 °C

C. 81.0 °C

D. 81.5 °C

Correct Answer: C. 81.0 °C

Question 8

Which report section should primarily explain discrepancies between experimental and theoretical results?

A. Title

B. Discussion

C. References

D. Appendix only

Correct Answer: B. Discussion

Question 9

What does blocking accomplish in an experiment?

A. It groups observations sharing a known source of variation

B. It removes the need for replication

C. It guarantees equal measurements

D. It replaces calibration

Correct Answer: A. It groups observations sharing a known source of variation

Question 10

For the relation h = Q̇/[A(Tₛ − T∞), which quantity has units of W·m⁻²·K⁻¹?

A. Q̇

B. A

C. Tₛ − T∞

D. h

Correct Answer: D. h