Measurement, Scaling and Questionnaire Design

Introduction

Business research converts abstract managerial ideas into evidence that can support decisions. Constructs such as customer satisfaction, employee commitment, perceived service quality, brand trust and innovation capability cannot usually be observed directly. Researchers therefore use measurement: the systematic assignment of numbers or symbols to characteristics of objects, people, events or responses according to explicit rules.

Measurement is not simply counting. It involves deciding what a construct means, identifying observable indicators, selecting an appropriate scale, designing questions, and evaluating whether the resulting data are sufficiently reliable and valid. Poor measurement can produce precise-looking but misleading results. For example, an organisation may report an average satisfaction score of 4.2, yet that figure has little value if respondents interpreted the question differently or if the scale measured service speed rather than satisfaction.

This chapter explains the four measurement scales, three important scaling techniques, reliability and validity, and the design, pilot testing and refinement of questionnaires. The emphasis is on MBA-level application: selecting defensible measures for managerial research and recognising the consequences of measurement choices for analysis and decision-making.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish nominal, ordinal, interval and ratio measurement scales.
2. Select an appropriate measurement scale for a managerial research variable.
3. Construct and evaluate Likert, semantic differential and Guttman scales.
4. Explain reliability and validity and identify common threats to each.
5. Design, pilot-test and refine a professional questionnaire.
6. Make ethically responsible decisions about respondent burden, privacy and data quality.

1. Measurement as the Operationalisation of Constructs

A construct is an abstract concept created for a research purpose. Examples include organisational commitment, perceived value and entrepreneurial orientation. An operational definition specifies how the construct will be observed or measured. For instance, customer loyalty might be operationalised through repurchase intention, recommendation intention and resistance to switching.

A useful measurement process has five stages:

1. Define the construct conceptually. Review theory and prior research to establish its boundaries.
2. Identify dimensions. A construct may be one-dimensional or multidimensional. Service quality, for example, may involve reliability, responsiveness and assurance.
3. Generate indicators. Indicators are observable statements, behaviours or attributes that represent the dimensions.
4. Choose response formats and scales. The scale should match the construct and the planned analysis.
5. Assess measurement quality. Reliability, validity and respondent comprehension must be examined before substantive conclusions are drawn.

Measurement can be direct or indirect. Monthly sales revenue is a relatively direct measure of a financial outcome. Perceived leadership effectiveness is indirect and normally requires several items. Multi-item measurement is often preferable for complex constructs because it captures more than one manifestation and reduces dependence on any single question.

A measurement instrument should also distinguish the unit of analysis from the unit of observation. In a study of organisational culture, the organisation may be the unit of analysis, while individual employees provide the observations. Aggregating individual responses requires theoretical and statistical justification; an employee’s perception should not automatically be treated as an objective organisational attribute.

2. Measurement Scales

The four classical scales form a hierarchy of information. Each higher level contains the properties of the levels below it and adds a stronger meaning to numerical differences. The hierarchy is useful, but researchers must not assume that a more complex scale is always better. The correct scale depends on the attribute and the rules used to measure it.

2.1 Nominal Scale

A nominal scale classifies observations into mutually exclusive and collectively exhaustive categories. The numbers assigned to categories are labels, not quantities. If “1” represents retail and “2” represents manufacturing, manufacturing is not twice retail.

Examples include gender categories, region, customer type, product category and yes/no responses. Appropriate summaries include frequencies, percentages, mode and cross-tabulations. The most suitable statistical tests depend on the research design, but arithmetic operations such as calculating a meaningful average of department codes are inappropriate.

Good nominal categories must be clearly defined. They should not overlap: a respondent should not fit two categories at once. They should also cover all plausible answers, using an “other, please specify” option where necessary. Sensitive demographic questions require inclusive and context-appropriate categories.

2.2 Ordinal Scale

An ordinal scale places observations in an order, but the distances between adjacent positions are not known to be equal. A ranking of five brands identifies first through fifth place, but it does not show whether the gap between first and second is the same as the gap between fourth and fifth.

Examples include class rank, agreement categories and perceived performance levels such as poor, fair, good and excellent. The median and percentiles are generally defensible summaries. Means are sometimes reported for ordered response categories, particularly for multi-item attitude scales, but this practice should be justified rather than treated as automatically valid.

Rank data also have practical limitations. Asking respondents to rank ten alternatives can impose substantial cognitive burden, and a rank forces trade-offs even when respondents view several alternatives as equally attractive. Rating scales may be preferable when independent evaluations are required.

2.3 Interval Scale

An interval scale has ordered values with equal intervals, but its zero point is arbitrary rather than a genuine absence of the attribute. Temperature measured in Celsius illustrates the principle: the difference between 10°C and 20°C equals the difference between 20°C and 30°C, but 20°C is not twice as hot as 10°C.

Interval-level data support addition and subtraction, means and standard deviations, subject to the assumptions of the selected statistical procedures. In business research, many psychological scales are treated as approximately interval when several well-designed items are combined and the response categories are equally spaced by design. This is a modelling decision, not a universal fact about all rating data.

2.4 Ratio Scale

A ratio scale has all the properties of the other scales and a meaningful, non-arbitrary zero indicating the absence of the measured quantity. Examples include sales revenue, number of complaints, delivery time, market share expressed as a proportion and number of employees. Ratio statements are meaningful: a firm with revenue of $20 million has twice the revenue of a firm with $10 million, assuming the same currency and accounting basis.

Ratio data support addition, subtraction, multiplication and division. However, the existence of a numerical zero does not by itself guarantee ratio measurement. The researcher must establish that the zero represents absence and that units are comparable. For example, calendar year has a numerical origin but is not a ratio variable.

2.5 Choosing a Scale

Scale selection should follow the construct rather than the researcher’s preferred statistical technique. Ask:

• What property is being measured?
• Are categories merely different, or can they be ordered?
• Are differences between values equal in meaning?
• Does zero represent a complete absence?
• What level of measurement is needed for the intended analysis?

A firm may measure customer type nominally, service-quality ratings ordinally or through a multi-item attitude scale, and annual expenditure on a ratio scale. Treating all three variables as if they had the same properties would distort interpretation.

3. Scaling Techniques for Attitudes and Perceptions

A scaling technique specifies how respondents express the intensity, direction or pattern of a characteristic. The following techniques are especially useful in business questionnaires.

3.1 Likert Scale

A Likert scale measures agreement with a series of statements. A typical item uses five or seven ordered response categories, such as:

“Employees of this company respond promptly to customer requests.”

1 = Strongly disagree; 2 = Disagree; 3 = Neither agree nor disagree; 4 = Agree; 5 = Strongly agree.

The individual statement is a Likert-type item. A Likert scale more properly refers to a set of related items combined to measure a construct. If all items are coded in the same direction, a composite score may be calculated as a sum or mean. For k items, the respondent’s mean score can be represented as:

Here, M is the respondent’s composite mean and xᵢ is the coded response to item i. This calculation assumes that the items represent the same construct, are coded consistently, and that the researcher has decided how to handle missing responses. A reverse-worded item must be recoded before aggregation. For a five-point item, reverse coding can use:

where x is the original response and xᵣ is the recoded response. The constant 6 arises because the minimum and maximum are 1 and 5; for a seven-point scale, the corresponding constant is 8.

Advantages of Likert scales include ease of administration, suitability for attitudes and perceptions, and the ability to improve coverage through multiple items. Limitations include acquiescence bias, central-tendency bias, social desirability and possible confusion caused by negatively worded items. Balanced wording and clear instructions can reduce these problems, although excessive use of reverse-worded items may itself confuse respondents.

The number of categories should reflect respondents’ ability to distinguish meaningful differences. Five categories are simple and accessible; seven may provide greater discrimination for informed respondents. Adding categories does not automatically improve validity.

3.2 Semantic Differential Scale

A semantic differential measures the meaning associated with an object by placing bipolar adjectives at opposite ends of a numerical continuum. For example:

The adjectives must be genuinely opposite and relevant to the object. “Modern–traditional,” “professional–unprofessional” and “valuable–worthless” may be appropriate in some contexts, but their meaning should be confirmed through pretesting. The scale is useful for brand image, service evaluation and comparing perceived profiles across competitors.

A semantic differential differs from a Likert scale in both wording and response task. Likert respondents indicate agreement with statements; semantic differential respondents position an object between bipolar descriptors. Researchers should explain the direction of scoring because some positive poles appear on the left and others on the right.

3.3 Guttman Scale

A Guttman, or cumulative, scale is designed so that agreement with a more demanding item implies agreement with less demanding items. Consider adoption of a digital service:

1. I have heard of the service.
2. I have visited the service website.
3. I have registered for the service.
4. I have completed a transaction using the service.

The intended cumulative pattern is 0000, 1100, 1110 or 1111, where 1 indicates endorsement. A respondent who has completed a transaction would normally be expected to have performed the earlier actions. In practice, respondents may produce exceptions, so the researcher must assess whether the cumulative structure is sufficiently consistent.

Guttman scaling is valuable when a latent progression or hierarchy is theoretically plausible, such as stages of adoption, increasing organisational commitment or levels of policy implementation. It should not be used merely because items can be arranged from easy to difficult. Items must represent a single underlying dimension, and empirical testing is needed to identify deviations from the expected pattern.

4. Reliability and Validity

Measurement quality has two central dimensions. Reliability concerns consistency; validity concerns whether the instrument measures what it is intended to measure. A measure can be reliable but invalid. A bathroom scale that consistently adds five kilograms is reliable in a narrow sense but not valid for measuring true weight.

4.1 Reliability

Reliability may be assessed in several ways:

• Test–retest reliability: administer the same instrument to the same respondents on two occasions and examine stability. It is appropriate for relatively stable constructs, but a long interval may allow genuine change and a short interval may create memory effects.
• Inter-rater reliability: compare judgments made by different observers or coders. It is essential when responses involve subjective classification.
• Internal consistency: examine whether items intended to measure one construct produce consistent responses. Cronbach’s alpha is commonly reported, although it is not a proof of unidimensionality or validity.

For k items, a common expression for coefficient alpha is:

Here, α is Cronbach’s alpha, k is the number of items, σᵢ² is the variance of item i, and σₜ² is the variance of the total score formed from the items. Alpha assumes that items are sufficiently related and are intended to represent one construct; it is affected by the number of items and should not be interpreted as a universal pass–fail rule. Very high alpha may indicate redundant wording rather than excellent measurement.

Reliability can be improved by clarifying items, removing ambiguous or poorly performing items, adding relevant indicators and standardising administration. Removing an item solely to increase alpha is inappropriate if it damages content coverage.

4.2 Validity

Validity is the degree to which evidence supports the interpretation and use of scores for their intended purpose. It is not a permanent property of a questionnaire independent of context. Important forms of evidence include:

• Face validity: whether the instrument appears, on inspection, to measure the intended construct. It is useful for respondent acceptance but is weak evidence by itself.
• Content validity: the extent to which items adequately cover the construct’s conceptual domain. Subject-matter experts and a systematic literature review are important.
• Criterion-related validity: the association between a measure and a relevant external criterion. Predictive validity concerns future outcomes; concurrent validity concerns a criterion measured at approximately the same time.
• Construct validity: evidence that the measure behaves as theory predicts. Convergent validity is supported when related measures correlate; discriminant validity is supported when distinct constructs are not excessively correlated.

Exploratory factor analysis may help examine whether items group into expected dimensions, while confirmatory factor analysis can test a theoretically specified measurement model. Factor analysis does not, by itself, establish conceptual validity; statistical patterns must be interpreted alongside theory and content evidence.

Common threats include vague wording, leading questions, common-method bias, social desirability, inadequate coverage, cultural misunderstanding and inappropriate translation. Validity is therefore cumulative: no single statistic can replace theoretical reasoning, expert review, cognitive testing and empirical examination.

5. Questionnaire Design

A questionnaire is a structured instrument for collecting standardised information. Its quality depends on both individual questions and the overall respondent experience.

5.1 Begin with the Research Purpose

Every question should serve a research objective, hypothesis, measurement requirement or screening decision. Create a measurement blueprint before writing items. The blueprint links each research question to constructs, dimensions, indicators, response format and planned analysis. Questions that will not be analysed or used for eligibility should normally be removed.

5.2 Write Clear Questions

Effective questions:

• use familiar, precise language;
• ask about one idea at a time;
• specify a reference period where relevant;
• identify the unit of analysis;
• avoid assumptions about knowledge or behaviour;
• provide balanced and exhaustive response options.

Avoid double-barrelled questions such as “How satisfied are you with the speed and courtesy of delivery?” Speed and courtesy are different attributes. Avoid leading wording such as “How satisfied are you with our excellent service?” and vague terms such as “regularly” unless the term is defined.

A problematic question may be split into two items:

1. How satisfied are you with the speed of delivery?
2. How satisfied are you with the courtesy of delivery staff?

The researcher should also decide whether to use open-ended or closed-ended questions. Closed questions facilitate coding and comparison; open questions can reveal unexpected issues but require more respondent effort and systematic coding.

5.3 Organise the Instrument

A practical order is:

1. introduction, purpose, estimated completion time and consent information;
2. eligibility and screening questions;
3. easy, relevant substantive questions;
4. more detailed evaluations or sensitive questions;
5. demographics and classification questions;
6. closing thanks and contact information.

Filter or skip logic should be tested carefully. Respondents should not be asked about an experience they have not had. Instructions must be visible and consistent across paper, web and mobile formats.

5.4 Address Response Bias and Ethics

Respondents may select socially desirable answers, agree with statements, choose the midpoint, rush, or abandon a long instrument. Reduce these risks by assuring confidentiality where possible, explaining the legitimate purpose of the study, using neutral wording, keeping the questionnaire proportionate and allowing “don’t know” or “not applicable” when substantively appropriate.

Ethical design requires informed consent, voluntary participation, data minimisation and secure handling of personal information. Collect only information necessary for the research purpose. Sensitive items should have a clear justification, and participants should be able to decline where feasible. Digital questionnaires must communicate how responses, identifiers, cookies and contact details will be used. Accessibility should be considered through readable layouts, adequate contrast, keyboard compatibility and language appropriate to the target population.

6. Pilot Testing and Refinement

A questionnaire should not be deployed immediately after drafting. Pilot testing identifies problems in interpretation, sequence, technical operation, timing and data coding.

Expert review asks researchers or subject specialists to assess coverage, relevance and wording. Cognitive interviewing explores how respondents understand a question, retrieve information, judge an answer and select a response. Techniques such as think-aloud interviewing and probing can reveal that a seemingly simple term has multiple meanings. A pilot survey with respondents resembling the target population tests completion time, missing data, response distributions, reliability and skip logic under realistic conditions.

A disciplined refinement process is:

1. record each identified problem;
2. classify it as conceptual, wording, response-format, layout or technical;
3. revise the instrument;
4. retest changes that may create new problems;
5. document decisions and version control;
6. freeze the final instrument before full data collection.

Pilot results must be interpreted cautiously. A small pilot is mainly diagnostic and is not normally a basis for definitive population estimates. A low response variation may indicate a homogeneous sample, a poorly discriminating item or a genuinely shared view. Reliability statistics from a small pilot are unstable, so qualitative evidence remains important.

7. Applied Case: Measuring Employee Experience in a Hybrid Firm

Background

Illustrative case: Northstar Analytics, a 600-employee consulting firm, introduced hybrid work and wanted to understand whether the policy improved employee experience and retention.

Problem

The initial management survey asked, “Are you satisfied with communication, flexibility and productivity under hybrid work?” The item was double-barrelled, assumed respondents could evaluate productivity objectively, and did not distinguish policy design from implementation. Management also proposed reporting a single average based on department codes and satisfaction categories.

Analysis

The research team created a measurement blueprint with three constructs: perceived flexibility, communication effectiveness and intention to remain. Each construct received four or five theoretically relevant Likert items. Department was retained as a nominal classification variable, while days worked remotely per week was measured as a ratio variable with a meaningful zero. The team included “not applicable” for employees whose roles could not be performed remotely.

Experts reviewed content coverage. Cognitive interviews revealed that “communication” meant manager updates to some employees and team coordination to others, so items were separated. A semantic differential section assessed the perceived character of the hybrid policy using pairs such as “unpredictable–predictable” and “unfair–fair.”

Approach and Refinement

A pilot with 45 employees found that one item used the ambiguous term “often.” It was replaced with a specified four-week reference period. One reverse-worded item produced inconsistent interpretations and was removed. Internal consistency was examined, but the team retained an item that was theoretically important despite a modest item correlation, planning further validation rather than mechanically maximising alpha. Skip logic was tested for contractors and employees without remote-work eligibility.

Outcome

The final instrument separated constructs, reduced respondent burden and generated scores that managers could interpret more responsibly. Analysis showed that flexibility was rated positively, but communication effectiveness varied substantially by team. The organisation therefore avoided claiming that hybrid work was universally successful and instead targeted manager communication training and team-level coordination practices.

Learning Points

The case demonstrates that questionnaire quality begins with construct definition, not formatting. It also shows why measurement levels must be respected, why multi-item scales require reliability and validity evidence, and why pilot testing can reveal conceptual problems that statistical analysis alone cannot detect.

8. Contemporary and Managerial Considerations

Online platforms make branching, randomisation, multimedia prompts and real-time quality monitoring possible. These features can reduce burden, but randomisation may complicate comparisons and automated checks may exclude legitimate respondents. AI tools can assist with draft translation, duplicate-item detection or readability review, but generated wording requires expert evaluation for cultural bias, construct drift, privacy risk and inappropriate inference. AI should support—not replace—researcher accountability.

Managers should treat measurement as an investment in decision quality. A dashboard containing highly reliable scores can still mislead if the construct is invalid, the sample is unrepresentative or the response process is biased. Results should therefore report the instrument, scale anchors, missing-data treatment, evidence of reliability and validity, and the limits of interpretation.

Challenges and Limitations

Measurement is affected by cultural differences in scale use, language equivalence, changing meanings of constructs and context effects. Respondents may interpret the same midpoint differently across countries or organisational groups. Translation should use forward translation, independent review and, where feasible, back-translation and cognitive testing. Measurement invariance analysis may be needed before comparing groups; without it, observed differences may reflect the instrument rather than the construct.

Another limitation is the danger of reducing rich managerial phenomena to a single score. Composite indices are useful for monitoring, but they may conceal important dimensions and unequal experiences. Researchers should preserve theoretically meaningful subscales and examine distributions, not only averages.

Chapter Summary

Measurement operationalises abstract constructs through explicit rules and observable indicators. Nominal scales classify, ordinal scales order, interval scales represent equal differences without a true zero, and ratio scales add a meaningful zero and support ratio statements. Likert scales measure agreement with statements and are commonly combined into multi-item constructs; semantic differential scales use bipolar adjectives to assess perceived meaning; and Guttman scales represent theoretically cumulative stages.

Reliability concerns consistency and may involve test–retest, inter-rater or internal-consistency evidence. Validity concerns the defensibility of score interpretations and requires content, criterion-related and construct evidence. Reliability does not guarantee validity. Effective questionnaires are aligned with research objectives, use clear and neutral wording, provide appropriate response options, manage respondent burden and protect privacy. Expert review, cognitive interviewing and pilot surveys are essential for detecting and correcting problems before full data collection.

Key Terms

Construct: An abstract concept investigated in research.

Operational definition: A specification of how a construct will be observed or measured.

Indicator: An observable item or behaviour representing a construct.

Nominal scale: A scale that classifies observations without numerical order.

Ordinal scale: A scale that orders observations without guaranteeing equal intervals.

Interval scale: A scale with equal differences but no meaningful absolute zero.

Ratio scale: A scale with equal differences and a meaningful zero.

Likert item: A statement rated by degree of agreement.

Likert scale: A set of related Likert-type items measuring a construct.

Semantic differential: A scale using bipolar adjectives and a response continuum.

Guttman scale: A cumulative scale in which stronger endorsements imply weaker endorsements.

Reliability: Consistency or stability of measurement.

Validity: Evidence supporting the intended interpretation and use of scores.

Content validity: Coverage of the relevant conceptual domain.

Construct validity: Evidence that a measure behaves as theory predicts.

Internal consistency: Similarity of responses among items intended to measure one construct.

Pilot test: A small-scale trial used to identify instrument problems.

Cognitive interview: An interview examining how respondents understand and answer questions.

Common-method bias: Bias arising because variables are measured through the same method or source.

Review Questions

Short Answer Questions

1. What distinguishes an interval scale from a ratio scale?
2. Why should department codes not be averaged?
3. What is the difference between a Likert-type item and a Likert scale?
4. Name two forms of validity evidence.
5. What is the purpose of cognitive interviewing?

Long Answer Questions

1. Explain the four measurement scales and illustrate each with a business example.
2. Compare Likert, semantic differential and Guttman scaling techniques, including their appropriate applications and limitations.
3. Explain why a reliable instrument may nevertheless be invalid.
4. Develop a questionnaire-design process for measuring customer loyalty.
5. Discuss the ethical and methodological issues involved in collecting employee attitudes through an online questionnaire.

Analytical / Application Questions

1. A bank wants to compare perceived trust across four customer segments using a six-item scale. Design a measurement and validation plan, including scale format, pilot testing and group-comparison safeguards.
2. A manager proposes measuring service quality with one question: “How satisfied are you with the speed, accuracy and friendliness of our service?” Diagnose the problems and redesign the measurement instrument.
3. A researcher reports an alpha of 0.95 for a ten-item employee engagement scale but has no evidence of content or construct validity. Evaluate the claim that the scale is excellent and recommend further analysis.

Multiple Choice Questions with Answers

Question 1

Which measurement scale permits meaningful statements such as one firm having twice the revenue of another?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: D. Ratio

Question 2

What does a Likert-type item normally ask respondents to indicate?

A. Agreement with a statement

B. A rank among competitors

C. A bipolar adjective position only

D. A numerical quantity with a true zero

Correct Answer: A. Agreement with a statement

Question 3

Which technique uses opposite adjectives such as reliable–unreliable?

A. Nominal classification

B. Semantic differential

C. Guttman scaling

D. Test–retest scaling

Correct Answer: B. Semantic differential

Question 4

A Guttman scale is based primarily on what principle?

A. Random ordering of categories

B. Equal numerical intervals

C. Cumulative endorsement of increasingly demanding items

D. Use of an arbitrary zero

Correct Answer: C. Cumulative endorsement of increasingly demanding items

Question 5

Which statement about reliability is correct?

A. Reliability proves validity

B. Reliability concerns consistency of measurement

C. Reliability applies only to ratio data

D. Reliability eliminates sampling error

Correct Answer: B. Reliability concerns consistency of measurement

Question 6

What is content validity primarily concerned with?

A. Whether items cover the construct domain

B. Whether scores predict future sales only

C. Whether respondents answer quickly

D. Whether categories have a true zero

Correct Answer: A. Whether items cover the construct domain

Question 7

Which questionnaire item is double-barrelled?

A. How many years have you worked here?

B. Which department employs you?

C. How satisfied are you with delivery speed and staff courtesy?

D. Have you used the service in the last month?

Correct Answer: C. How satisfied are you with delivery speed and staff courtesy?

Question 8

What is the main purpose of a pilot survey?

A. To guarantee population validity

B. To identify wording, timing and operational problems

C. To replace the main sample

D. To prove causal relationships

Correct Answer: B. To identify wording, timing and operational problems

Question 9

For a five-point item coded from 1 to 5, which expression reverse-codes response x?

A. xᵣ = x + 5

B. xᵣ = 5 − x

C. xᵣ = 6 − x

D. xᵣ = x ÷ 5

Correct Answer: C. xᵣ = 6 − x

Question 10

A reliable but invalid instrument is best described as one that:

A. Produces consistent scores but measures the wrong construct

B. Produces random scores but measures the correct construct

C. Has no response categories

D. Uses only ratio variables

Correct Answer: A. Produces consistent scores but measures the wrong construct