Measurement, Evaluation and PR Analytics

Introduction

Public relations (PR) is accountable not only for what an organisation communicates, but also for what communication achieves. A press release, media interview, event, website article, or social-media campaign is an output. It is not automatically evidence of understanding, trust, behaviour change, reputation improvement, or organisational value. Measurement and evaluation provide the disciplined processes through which practitioners connect communication activity to objectives and decisions.

This distinction is especially important in a media environment dominated by dashboards and real-time data. Digital platforms can report impressions, clicks, views, reactions, and followers within seconds. However, easily counted activity is not necessarily meaningful evidence. A large number of impressions may result from repeated exposure to the same people; a high engagement rate may reflect controversy; positive sentiment software may misclassify sarcasm; and a rise in website traffic may not indicate that audiences understood or acted on a message.

Professional evaluation therefore begins with objectives, not with available metrics. It combines quantitative evidence—such as reach, survey scores, web behaviour, and conversions—with qualitative evidence, including interviews, message analysis, stakeholder feedback, and observation. The aim is not merely to demonstrate that PR was busy, but to determine whether communication contributed to desirable outcomes and organisational impact.

Learning Objectives

By the end of this chapter, students should be able to:

1. Distinguish outputs, outtakes, outcomes, and impact.
2. Translate PR objectives into measurable indicators and key performance indicators (KPIs).
3. Design a basic evaluation framework and dashboard.
4. Interpret media, web, social, survey, sentiment, and share-of-voice data critically.
5. Apply recognised evaluation principles, including the Barcelona Principles.
6. Identify misleading metrics, attribution problems, bias, and ethical risks.
7. Use evidence to improve communication strategy and managerial decision-making.

1. The Logic of PR Evaluation

Evaluation is the systematic assessment of the design, implementation, and effects of a communication programme. Measurement is the collection and description of evidence; evaluation adds interpretation and judgement. Research supplies the evidence and methods on which both depend.

A useful logic chain is:

Inputs → Activities → Outputs → Outtakes → Outcomes → Impact

Inputs include staff time, budget, research, technology, and organisational support. Activities are the communication actions undertaken, such as pitching journalists, publishing content, conducting stakeholder meetings, or running an advocacy campaign.

Outputs

Outputs are the immediate, countable products of PR activity. Examples include:

• number of media releases distributed;
• number of media briefings conducted;
• media placements obtained;
• potential audience or estimated reach;
• social posts published;
• event attendance;
• videos produced;
• website articles published.

Outputs answer: What did the organisation do, and what communication material was produced? They are useful for monitoring implementation, but they do not prove success.

Outtakes

Outtakes describe what audiences receive, notice, understand, remember, or feel about a message. Examples include awareness, message recall, comprehension, perceived credibility, and changes in knowledge or attitude.

Outtakes answer: What did the audience take from the communication? A survey question such as “Which of the following messages do you remember?” measures recall more directly than a count of social-media views.

Outcomes

Outcomes are changes in audience behaviour, intention, relationships, or stakeholder support. Examples include increased use of a public service, product enquiries, policy support, employee participation, donations, or reduced misinformation.

Outcomes answer: What changed among the relevant publics? They are closer to strategic objectives than outputs and outtakes.

Impact

Impact is the broader organisational, social, or public value associated with sustained outcomes. Examples include improved organisational reputation, stronger stakeholder relationships, greater legitimacy, improved public health, or reduced social harm.

Impact is often influenced by many factors beyond PR. Therefore, practitioners should avoid claiming that communication alone caused a long-term result unless the research design supports such a conclusion.

2. Objectives as the Foundation of Measurement

Evaluation becomes weak when objectives are vague. “Increase awareness” is incomplete unless the organisation specifies whose awareness, of what, by how much, and by when. Effective objectives are specific, measurable, achievable, relevant, and time-bound (SMART), although professional objectives should also identify the intended public and the desired organisational contribution.

A strong objective might be:

Among first-year students at City University, increase unaided awareness of the free mental-health counselling service from 30% to 45% by the end of the academic year, while increasing first appointments by 15%.

This objective contains a public, a baseline, a target, a deadline, an outtake, and an outcome. It also suggests appropriate methods: a baseline and follow-up survey for awareness, and administrative records for appointments.

A measurement plan should map each objective to indicators, data sources, timing, responsibility, and decision rules.

Indicators and KPIs

An indicator is an observable measure related to an objective. A KPI is an indicator selected as especially important for judging progress or performance. Not every metric should be a KPI. A dashboard containing dozens of numbers can obscure rather than improve decision-making.

Good KPIs are relevant to the objective, clearly defined, consistently measured, timely, and actionable. Each should have a documented numerator, denominator where relevant, data source, reporting period, owner, and interpretation rule.

For example, an engagement rate may be calculated as:

Here, engagements may include reactions, comments, shares, saves, or link clicks, but the organisation must state exactly which actions are included. Reach is the number of unique accounts or users shown the content, if the platform provides that measure. The formula is meaningful only when the same definitions and time period are used. Platform-reported reach may be estimated and should not automatically be treated as verified exposure.

For a conversion rate:

The denominator must match the decision being studied. For example, completed registrations divided by landing-page visitors is different from completed registrations divided by total campaign impressions.

3. Recognised Evaluation Principles

The Barcelona Principles, developed by the International Association for Measurement and Evaluation of Communication (AMEC) and subsequently updated, are widely used as a professional framework. Their central ideas include the following:

1. Goal setting and measurement are fundamental. Objectives should be defined before activity begins.
2. Measurement and evaluation should identify outputs, outcomes, and potential impact. Counting activity alone is insufficient.
3. Outcomes and impact should be measured whenever possible. The appropriate level depends on the objective and available evidence.
4. Communication measurement should include qualitative and quantitative analysis. Numbers require context and interpretation.
5. Advertising Value Equivalency (AVE) is not the value of public relations. The price of buying equivalent advertising space does not establish PR value.
6. Social and digital media should be measured consistently with other media. Channel novelty does not remove the need for sound definitions.
7. Measurement should be transparent, valid, reliable, and ethical. Methods, assumptions, limitations, and data sources should be reported.

The principles do not prescribe one universal metric. They encourage practitioners to select methods that fit the objective, public, channel, and stage of the logic chain.

4. Media and Message Evaluation

Media coverage

Media analysis can assess volume, prominence, reach, source quality, topic, key-message inclusion, spokesperson presence, tone, and accuracy. A simple coding scheme might classify each article by whether the organisation’s main message appears, whether the organisation is quoted, and whether the article is prominent in the publication.

Volume is useful for monitoring visibility, but it is not a sufficient measure of influence. One authoritative article read by a decision-making stakeholder may be more valuable than many low-relevance mentions.

Share of voice

Share of voice (SOV) estimates an organisation’s proportion of coverage or conversation relative to selected competitors or issues:

The denominator must be defined carefully. It may include competitors, all organisations in a sector, or all mentions of a public issue. Comparisons are valid only when the time period, media set, search terms, geography, and coding rules remain comparable. SOV does not measure reputation or favourability. An organisation can have a high share of negative coverage.

Sentiment

Sentiment analysis classifies content as positive, neutral, or negative, either through human coding, automated natural-language processing, or a combination. Automated systems can be useful for large datasets but may struggle with irony, local language, slang, mixed emotions, quoted speech, and context. A sentiment score should therefore be treated as an indicator, not as an objective emotional fact.

A basic net sentiment index may be represented as:

The result ranges from −1 to +1 when all mentions are coded into the categories and neutral content remains in the denominator. The formula is not universal; organisations must disclose category definitions and coding quality. Human validation of a sample is recommended.

Message pull-through

Message pull-through measures how often a desired message appears in relevant coverage or audience responses:

It is most useful when the communication objective concerns understanding or framing. It should not be interpreted as proof that audiences believed the message.

5. Web and Social Analytics

Digital analytics can connect communication exposure with online behaviour, but interpretation requires attention to user journeys, tracking quality, privacy, and attribution.

Important web measures include:

• Users or visitors: distinct users identified under the platform’s rules.
• Sessions: periods of interaction, defined by the analytics system.
• Page views: number of pages loaded, including possible repeat views.
• Engaged sessions: sessions meeting a platform-specific engagement definition.
• Referral source: the channel or site that directed traffic.
• Landing-page completion: completion of a desired action after arrival.
• Exit and abandonment: points at which users leave or fail to complete a process.

Social measures include reach, impressions, video completion, saves, shares, comments, click-throughs, follower growth, and response time. These are useful only when connected to a strategic purpose. For a public-information campaign, completed information downloads or service applications may be more meaningful than likes.

A funnel can help organise evidence:

Exposure → Attention → Understanding → Action → Sustained behaviour

Each stage requires different evidence. Impressions may indicate potential exposure; video completion may indicate attention; a knowledge survey may indicate understanding; registrations may indicate action; repeat usage or follow-up records may indicate sustained behaviour.

Attribution must be handled cautiously. A person may encounter a news article, search for the organisation, click a social post, and later convert after an email reminder. A “last-click” model assigns all credit to the final channel even though earlier contact may have been important. PR teams should report the attribution model and, where possible, use multiple methods such as tagged links, surveys asking “How did you hear about us?”, controlled geographic comparisons, or pre- and post-campaign research.

6. Surveys and Research Methods

Surveys are particularly valuable for measuring awareness, knowledge, recall, attitudes, trust, intention, and self-reported behaviour. A valid survey requires clear constructs and questions that actually measure them.

Poor question: “Did our excellent campaign successfully improve your trust?”

Better questions separate concepts:

• “Before today, had you heard of the service?”
• “How much do you agree that the organisation provides reliable information?”
• “Which message do you recall?”
• “Have you used the service during the past three months?”

Researchers should consider sampling, response rate, non-response bias, question order, social desirability, wording, and timing. A convenient online sample may be fast but may not represent the target public. If a sample is weighted, the weighting method should be documented.

A change in survey percentages can be described as a percentage-point change. If awareness rises from 30% to 45%, the change is 15 percentage points. The relative increase is:

These are different statements. The first is usually clearer for public reporting.

A pre-test/post-test design compares measures before and after a programme. It can show association with the campaign, but without a control group it cannot eliminate alternative explanations such as news events, seasonality, or policy changes. A stronger design uses a comparison group not exposed to the programme, while recognising that random assignment is often difficult in public communication.

7. Dashboards and Interpretation

A dashboard is a decision-support interface, not a collection of every available number. A good PR dashboard contains:

1. the objective and reporting period;
2. a small set of outcome-oriented KPIs;
3. baseline, target, current value, and trend;
4. segmentation by relevant public or channel;
5. explanatory qualitative evidence;
6. data-quality notes and limitations;
7. a recommended management action.

For example, a campaign dashboard might show awareness, message recall, qualified website actions, service applications, and relevant media quality. It might also show output data such as placements and reach, but clearly label these as activity indicators.

Colour coding can support rapid interpretation, but red/amber/green status should be based on pre-defined thresholds rather than subjective impressions. A dashboard should distinguish:

• descriptive metrics: what happened;
• diagnostic metrics: why it may have happened;
• predictive indicators: what may happen next;
• decision metrics: what action should be taken.

Correlation does not establish causation. If positive sentiment and donations rise in the same month, the relationship may reflect a fundraising season, a major news event, or an unrelated economic change. Evaluation reports should state the strength of evidence and avoid exaggerated causal language.

8. Worked Example: University Mental-Health Service Campaign

This illustrative case concerns a fictional university, Northbridge University. Student research showed that only 30% of first-year students knew that counselling appointments were free. The university wished to increase awareness and service use without implying that counselling was only for students in crisis.

Background and problem

The communication team had previously reported success using impressions and social-media likes. However, service managers could not determine whether students understood the service or whether communication reached students who needed it.

Objectives

The campaign set three objectives for one academic year:

• increase unaided awareness from 30% to 45%;
• increase accurate recall of the “free and confidential” message;
• increase first appointments by 15%, while monitoring waiting times and service capacity.

Approach

The team conducted a baseline survey, student focus groups, message testing, campus posters, student-media partnerships, short videos, residence-hall briefings, and a tagged landing page. It also trained peer ambassadors and established a data-sharing agreement with the counselling service using aggregated counts rather than identifiable student records.

Measurement plan

Outputs included 24 videos, 10 student-media stories, 15 briefings, and 40,000 estimated poster exposures. Outtakes were measured through a follow-up survey asking about awareness, message recall, perceived confidentiality, and perceived relevance. Outcomes were assessed through landing-page actions and aggregated first appointments. A comparison with the previous academic year helped identify unusual changes, though it did not provide a perfect control group.

Results

The follow-up survey found unaided awareness of 47%, up from 30%, and accurate recall of the free-and-confidential message among 41% of respondents, up from 22%. First appointments rose from 800 to 920:

The campaign therefore met the appointment target and exceeded the awareness target. However, the evaluation found that demand increased faster than appointment capacity, producing longer waiting times. The communication team could not claim that the campaign alone caused the increase because a new student-wellbeing policy and examination stress also occurred during the period.

Outcome and learning

The university retained the messages but revised its objective for the following year to include access quality: reducing the median waiting time while maintaining awareness. The case demonstrates why evaluation must examine consequences, not simply celebrate increased demand. A communication programme can be effective in generating action while creating operational pressure that requires managerial response.

9. Misleading Metrics and Common Errors

Several practices can create an inflated impression of success:

• treating potential reach as actual readership;
• counting duplicate impressions as unique people;
• equating likes with support or attitude change;
• using follower growth without checking relevance or authenticity;
• reporting AVE as PR value;
• presenting automated sentiment without validation;
• selecting only favourable data after the campaign;
• changing definitions between reporting periods;
• claiming causation from before-and-after movement alone;
• using a high percentage without reporting the denominator or sample size.

Vanity metrics are numbers that look impressive but provide little guidance for decisions. They are not always useless; reach or followers may be relevant at an early awareness stage. Their value depends on the objective and their relationship to later outcomes.

A useful test is: If this metric rises, what decision would change? If the answer is none, the metric may not belong on the main dashboard.

10. Ethics, Governance, and Privacy

PR analytics involves collecting and interpreting information about people. Ethical practice requires lawful processing, purpose limitation, data minimisation, secure storage, appropriate consent where required, and transparent communication about data use. Privacy rules vary by jurisdiction, but ethical responsibility extends beyond legal compliance.

Practitioners should avoid identifying vulnerable individuals, inferring sensitive characteristics without justification, or combining datasets in ways that enable re-identification. Social-media content that is publicly visible is not automatically ethically unrestricted for every analytical purpose.

Governance should specify data ownership, access rights, retention periods, vendor responsibilities, quality checks, and procedures for correcting errors. Automated tools should be monitored for bias. Sentiment and classification systems may perform poorly for minority languages, dialects, or communities whose communication styles differ from training data.

Reports should disclose uncertainty. A confidence interval, margin of error, coding reliability estimate, or data-quality warning can be more informative than a highly precise-looking number. Precision in presentation must not exceed precision in the evidence.

11. Technology, AI, and Contemporary Practice

Artificial intelligence can support media monitoring, topic clustering, transcription, anomaly detection, summarisation, and preliminary sentiment classification. It can reduce manual workload, but it does not remove the need for human judgement. Automated systems may hallucinate summaries, misattribute quotations, miss context, or reproduce bias in training data.

Responsible use includes defining the task, testing the system on representative samples, comparing automated results with human coding, documenting prompts and model versions, protecting confidential information, and requiring human review before publication or strategic decisions. AI-generated insight should be treated as provisional evidence until validated.

Contemporary measurement is also moving toward integrated evaluation. Rather than treating earned, owned, shared, and paid media as isolated channels, practitioners increasingly examine audience journeys across them. This integration is valuable only when definitions are harmonised and duplicate audiences are identified.

12. Best-Practice Evaluation Process

A practical evaluation process can follow these stages:

1. Clarify the organisational and communication objectives.
2. Identify publics, desired changes, baselines, and timeframes.
3. Build a logic model from activities to impact.
4. Select a balanced set of output, outtake, outcome, and impact indicators.
5. Define each metric and its data source before collection begins.
6. Choose suitable methods, combining quantitative and qualitative evidence.
7. Collect baseline data and establish comparison points where feasible.
8. Check data quality, privacy, sampling, coding, and platform limitations.
9. Interpret findings in relation to context and alternative explanations.
10. Report limitations and convert evidence into decisions.
11. Review the framework after the programme and improve future objectives.

The professional value of evaluation lies in learning and accountability. A campaign that misses a target can still produce valuable evidence if the organisation understands why and changes its approach. Conversely, a campaign that exceeds an output target but fails to alter relevant outcomes should not be described as fully successful.

Chapter Summary

PR measurement begins with objectives and follows a logic chain from inputs and activities to outputs, outtakes, outcomes, and impact. Outputs document what communication produced; outtakes assess audience awareness, understanding, and response; outcomes examine changes in behaviour, attitudes, or relationships; and impact concerns broader organisational or social value.

KPIs should be relevant, defined consistently, and connected to decisions. Media analysis, share of voice, sentiment, web analytics, social metrics, and surveys each provide useful but limited evidence. No single metric proves PR success. Dashboards should combine outcome indicators with carefully labelled activity measures, qualitative context, baseline comparisons, and limitations.

The Barcelona Principles emphasise objectives, outcomes, transparency, valid methods, and the rejection of AVE as a substitute for PR value. Ethical evaluation protects privacy, recognises bias, validates automation, and avoids unsupported causal claims. The central professional discipline is to measure what matters rather than merely what is easy to count.

Key Terms

Advertising Value Equivalency (AVE): A monetary estimate based on the cost of comparable advertising space; not a valid substitute for PR value.

Attribution: The process of estimating how communication channels contributed to an action or outcome.

Barcelona Principles: Recognised professional principles for ethical, transparent, and meaningful communication measurement and evaluation.

Baseline: The measurement of a condition before a communication programme begins.

Dashboard: A structured visual display of selected measures used for monitoring and decision-making.

Engagement rate: Engagements expressed as a proportion of a defined audience base such as reach.

Evaluation: Systematic assessment of communication implementation, effects, value, and learning.

Impact: Broader and often longer-term organisational, social, or public value.

Indicator: An observable measure related to an objective.

Key performance indicator (KPI): A strategically important indicator used to judge progress or performance.

Message pull-through: The proportion of relevant content containing a desired key message.

Outcomes: Changes in audience behaviour, attitudes, relationships, or support.

Outtakes: What audiences notice, understand, remember, or feel about communication.

Outputs: Immediate products or activities generated by PR work.

Reach: The number of unique users or accounts potentially exposed to content, according to a defined measurement system.

Sentiment analysis: Classification or interpretation of expressed evaluative tone in communication content.

Share of voice: An organisation’s proportion of relevant mentions or coverage within a defined set.

Vanity metric: A measure that appears impressive but has limited strategic or decision-making value.

Review Questions

Short Answer Questions

1. Distinguish between outputs, outtakes, outcomes, and impact, giving one example of each.
2. Why should PR objectives be defined before metrics are selected?
3. What is share of voice, and why does it not measure reputation by itself?
4. Identify two limitations of automated sentiment analysis.
5. Why is AVE not considered a valid measure of PR value?

Long Answer Questions

1. Explain how the Barcelona Principles influence the design of a PR evaluation programme.
2. Discuss how web and social analytics can be connected to a communication funnel from exposure to sustained behaviour.
3. Compare pre-test/post-test surveys with designs that use a comparison group.
4. Explain the characteristics of an effective PR dashboard and distinguish descriptive, diagnostic, and decision metrics.
5. Discuss the ethical and governance issues involved in using AI and personal data for PR analytics.

Analytical / Application Questions

1. A campaign receives 2,000,000 impressions, 80,000 engagements, and 20,000 landing-page visits. Explain which metrics are outputs, which may indicate attention or action, and what additional evidence is needed before claiming campaign success.
2. An organisation’s positive mentions rise from 400 to 600, but total coded mentions rise from 1,000 to 3,000. Calculate net sentiment for each period using the formula in the chapter, assuming negative mentions are 200 in the first period and 900 in the second. Interpret the change cautiously.
3. Design a measurement framework for a public-health campaign that aims to increase awareness, correct misinformation, and increase use of a preventive service. Include one output, one outtake, one outcome, one impact indicator, data sources, and key limitations.

Multiple Choice Questions with Answers

Question 1

Which measure is most clearly an outtake?

A. Number of media releases distributed

B. Number of campaign videos produced

C. Audience recall of the campaign’s main message

D. Number of service applications completed

Correct Answer: C. Audience recall of the campaign’s main message

Question 2

What is the main limitation of using potential reach as proof of PR success?

A. It cannot be calculated as a percentage

B. It does not establish that people actually noticed or understood the message

C. It measures only negative coverage

D. It is always based on survey data

Correct Answer: B. It does not establish that people actually noticed or understood the message

Question 3

If an organisation receives 120 relevant mentions out of 600 total relevant mentions, its share of voice is:

A. 5%

B. 12%

C. 20%

D. 50%

Correct Answer: C. 20%

Question 4

Which principle is associated with the Barcelona Principles?

A. AVE is the definitive measure of PR value

B. Outputs alone are sufficient for evaluation

C. Objectives should be set before measurement

D. Qualitative evidence should be excluded

Correct Answer: C. Objectives should be set before measurement

Question 5

A campaign has 500 engagements and a reach of 10,000. Using the chapter’s engagement-rate formula, the engagement rate is:

A. 0.5%

B. 5%

C. 20%

D. 50%

Correct Answer: B. 5%

Question 6

Which is the strongest reason to use a comparison group in campaign evaluation?

A. It guarantees that all respondents answer honestly

B. It helps assess whether changes may reflect factors other than the campaign

C. It eliminates the need for a baseline

D. It measures only media output

Correct Answer: B. It helps assess whether changes may reflect factors other than the campaign

Question 7

Why should automated sentiment results be human-validated?

A. Algorithms cannot count any words

B. Automated systems may misinterpret sarcasm, context, or quoted speech

C. Human coding is always faster

D. Sentiment has no relationship to communication

Correct Answer: B. Automated systems may misinterpret sarcasm, context, or quoted speech

Question 8

A rise in awareness from 30% to 45% represents which change?

A. 15 percentage points and a 50% relative increase

B. 15% and a 30% relative increase

C. 45 percentage points and a 15% relative increase

D. 50 percentage points and a 15% relative increase

Correct Answer: A. 15 percentage points and a 50% relative increase

Question 9

Which metric is most directly an outcome for a counselling-service campaign?

A. Number of posters designed

B. Number of social posts published

C. Number of first appointments completed

D. Estimated poster exposures

Correct Answer: C. Number of first appointments completed

Question 10

Which practice best supports ethical PR analytics?

A. Collecting all available personal data indefinitely

B. Reporting only favourable metrics

C. Documenting definitions, limitations, privacy safeguards, and data sources

D. Allowing an AI system to make final strategic decisions without review

Correct Answer: C. Documenting definitions, limitations, privacy safeguards, and data sources