Communication Analytics, AI and Future Practice

Introduction

Leadership communication increasingly occurs in environments where messages can be counted, compared and analysed. Email systems provide delivery and response data; collaboration platforms show participation; employee surveys reveal perceptions; and network analysis can identify patterns of connection across teams. Artificial intelligence (AI) can assist with drafting, translation, summarisation and analysis. These capabilities create opportunities for more responsive leadership, but they also create risks: metrics may be mistaken for meaning, algorithms may reproduce bias, confidential information may be exposed, and synthetic media may undermine trust.

Communication analytics is therefore not simply the collection of digital traces. It is the disciplined use of evidence to understand communication processes and improve leadership decisions. Effective practice connects a clearly defined objective to appropriate indicators, reliable data, careful interpretation and an ethical response. The central question is not “How much communication occurred?” but “Did communication help the intended people understand, participate, coordinate or act?”

Learning objectives

By the end of this chapter, students should be able to:

1. Define communication objectives and distinguish outputs, outcomes and impacts.
2. Select suitable indicators for surveys, sentiment, engagement and communication networks.
3. Interpret quantitative and qualitative evidence without overstating conclusions.
4. Evaluate communication outcomes using valid comparisons and feedback loops.
5. Use generative AI responsibly for drafting, analysis and accessibility.
6. Identify risks involving bias, verification, privacy, security, deepfakes and human oversight.
7. Integrate evidence and responsible AI into a practical leadership communication plan.

1. From communication activity to organisational value

A communication objective describes the intended change in knowledge, belief, behaviour, relationship or coordination. It should be specific enough to guide action and evaluation. “Improve communication” is too broad. “Within eight weeks, frontline supervisors will be able to explain the three safety changes and know where to report an incident” is more useful because it identifies an audience, outcome and timeframe.

A useful results chain is:

Inputs → Activities → Outputs → Outcomes → Impact

Inputs include time, staff, technology and budget. Activities include briefings, messages, listening sessions and coaching. Outputs are immediate products, such as messages sent, meetings held or resources accessed. Outcomes are changes in understanding, confidence, behaviour or relationships. Impact is the wider organisational consequence, such as safer work, improved retention or better service quality.

Outputs are usually easier to count than outcomes, but they are not substitutes for them. A message may be delivered to every employee without being understood or trusted. Similarly, high attendance at a town hall does not prove that leaders answered important questions.

Objectives and indicators

An indicator is an observable sign used to assess progress toward an objective. Good indicators are relevant, sufficiently reliable, feasible to collect and interpretable in context. A balanced measurement set normally contains:

A communication indicator should be linked to a decision. If leaders cannot explain what they would do when the indicator rises or falls, the indicator may be merely decorative.

Measurement concepts

For a proportion or rate, the general calculation is:

The denominator must be defined. For example, a response rate uses eligible invitees, not necessarily all employees:

If 180 of 600 invited employees complete a survey, the response rate is 30%. This describes participation, not satisfaction. Non-response may be systematic: employees with limited internet access, low trust or heavy workloads may be under-represented.

For change between two percentages, an absolute percentage-point change is:

where p₁ and p₂ are proportions expressed on the same scale. A change from 50% to 60% is 10 percentage points, or a 20% relative increase because (0.60 − 0.50) ÷ 0.50 = 0.20. These are different claims and should not be confused.

2. Designing a communication measurement system

A practical measurement system begins with a logic model and a baseline. The baseline records the situation before an intervention. Targets should be realistic and accompanied by a timeframe. Measurement plans should specify the audience, data source, collection frequency, responsible person, privacy safeguards and decision rule.

A simple planning template is:

Reliability, validity and comparison

Reliability concerns consistency. If the same stable condition produces very different results because of ambiguous questions or inconsistent coding, the measure is unreliable. Validity concerns whether the measure represents the intended construct. The number of messages sent may be reliable but has weak validity as a measure of trust.

Survey questions should be clear, neutral and limited to one idea. “The strategy was clear and inspiring” combines clarity and inspiration. Response scales should be labelled consistently. A survey should explain purpose, voluntary participation where applicable, confidentiality and how results will be used.

Comparisons require care. Compare like with like: the same population, question wording, timeframe and response conditions. A before-and-after change can be associated with a communication intervention, but it does not by itself prove causation. Other events—policy changes, media coverage, leadership turnover or economic pressure—may explain the result. Stronger evaluation may use a comparison group, repeated measures or mixed methods.

3. Interpreting surveys and engagement data

Surveys can measure perceived clarity, trust, psychological safety, relevance, confidence and willingness to act. They are especially useful when combined with open-text responses, interviews or observation. Quantitative results provide scale; qualitative evidence provides meaning.

The arithmetic mean is:

where xᵢ is an individual response and n is the number of responses. The mean is useful for approximately interval-scaled responses, but ordinal ratings such as “strongly disagree” to “strongly agree” should not automatically be treated as precise equal intervals. The median and distribution may be more informative, especially when responses are skewed.

For a sample standard deviation:

The n − 1 denominator is used when estimating population variability from a sample. Standard deviation describes dispersion, not quality. A high average with a wide spread may indicate that some groups are experiencing communication very differently.

Engagement is a broad construct and must be operationalised. It may include attention, participation, emotional connection, contribution and follow-through. Clicks, views or attendance are behavioural traces, not complete measures of engagement. A person may open a message because it is mandatory, while another may read carefully without clicking a link.

Useful engagement indicators include:

• participation rate, with a clearly defined denominator;
• completion or follow-through rate;
• questions asked and issues resolved;
• repeat participation across time;
• perceived relevance and ability to act;
• evidence of application in work practices.

Avoid ranking individuals using communication traces without a strong ethical and operational justification. Such ranking can create surveillance pressure and encourage superficial activity rather than meaningful communication.

4. Sentiment and text analysis

Sentiment analysis classifies text according to an affective orientation, such as positive, neutral or negative. More advanced systems identify emotions, topics or stance. It can help leaders process large volumes of comments, but sentiment is not the same as truth, importance or intention.

A statement such as “The rollout was ambitious, but the support was excellent” contains mixed sentiment. Sarcasm, cultural expression, professional language and domain-specific terms can be misclassified. A negative comment may indicate valuable risk information, while many positive comments may reflect politeness rather than genuine confidence.

A responsible workflow is:

1. Define the question and unit of analysis.
2. Obtain data lawfully and communicate its purpose.
3. Remove unnecessary personal identifiers.
4. Establish a coding scheme and test it on a sample.
5. Review errors across languages, groups and communication channels.
6. Combine automated classification with human reading.
7. Report uncertainty, themes and representative evidence.
8. Act and check whether the response improved the underlying issue.

If human coders classify a sample, agreement can be examined using Cohen’s kappa for two coders:

Pₒ is the observed proportion of agreement and Pₑ is the agreement expected by chance under the specified coding assumptions. Kappa is not a universal quality score: it is affected by category prevalence and coding design. It should be interpreted alongside examples, coder training and the purpose of the analysis.

5. Communication network analysis

Network analysis represents people or units as nodes and relationships or interactions as edges. A network may describe advice-seeking, information exchange, collaboration or message replies. The meaning of an edge must be defined; an email connection does not necessarily mean trust or useful collaboration.

Important concepts include:

• Degree centrality: the number of direct connections. In directed networks, in-degree and out-degree distinguish received from initiated ties.
• Betweenness centrality: the extent to which a node lies on shortest paths between other nodes. It can suggest brokerage, but it depends on how ties and paths are defined.
• Density: the proportion of observed ties among all possible ties under a specified network definition.
• Components and clusters: groups connected internally, which may reveal silos or communities.

For a simple undirected network with n nodes and no self-loops, the maximum number of possible edges is:

If 5 departments are treated as nodes and 6 of the 10 possible undirected relationships are present, density is 0.6, or 60%. This calculation assumes that every possible relationship is equally eligible, that ties are binary and undirected, and that multiple interactions are not weighted. Real organisational networks often require weighted, directed or temporal models.

Network evidence must be handled sensitively. A highly connected person may be a helpful coordinator, but may also be overloaded. A peripheral position may reflect role design rather than exclusion. Mapping individuals can expose confidential relationships or enable unfair surveillance. Aggregated team-level reporting, informed governance and strict access controls are preferable to unnecessary individual disclosure.

6. Evaluating outcomes and making decisions

Evaluation connects evidence to judgement. A useful evaluation asks four questions:

1. What changed?
2. For whom and under what conditions?
3. What other explanations are plausible?
4. What should leaders do next?

A mixed-method approach may combine a survey trend, open comments, interviews, network patterns and operational outcomes. Triangulation does not mean that all sources must agree. Disagreement can reveal important differences—for example, leaders may believe that messages are clear while frontline staff report uncertainty.

Leaders should distinguish correlation from causation. If message engagement and safety performance rise together, communication may have contributed, but training, supervision or seasonal conditions may also have changed. Evaluation is stronger when it establishes temporal order, uses a credible comparison, measures plausible alternatives and checks whether the mechanism makes sense.

A useful communication dashboard should be limited to decision-relevant indicators. It should show definitions, timeframe, population, missing data and confidence or uncertainty where appropriate. Red, amber and green labels can support attention, but they should not conceal the underlying distribution or encourage target gaming.

7. Generative AI in leadership communication

Generative AI produces text, images, audio, code or other content from prompts and learned patterns. It can support leadership communication in three main ways.

Drafting and transformation

AI can generate first drafts, alternative tones, plain-language versions, translations, summaries, agendas and questions for listening sessions. The leader remains responsible for purpose, accuracy, context, empathy and accountability. A fluent draft can still be factually wrong or culturally inappropriate.

Analysis

AI can cluster open comments, suggest themes, identify repeated questions and produce preliminary summaries. It should not be treated as an objective observer. Prompts, training data, model design and sampling choices influence results. Sensitive data should not be entered into an unapproved system.

Accessibility

AI can help produce captions, transcripts, audio versions, reading-level adaptations and translation. Accessibility requires human testing: captions may misrecognise names, translations may distort meaning, and simplified language may remove important qualifications. Accessibility is a communication responsibility, not merely an automated output.

A responsible AI workflow

A practical workflow has seven stages:

1. Define: specify the communication purpose, audience and acceptable use.
2. Protect: classify data and remove personal or confidential information unless an approved system and lawful basis exist.
3. Prompt: provide context, constraints and desired format without requesting unsupported certainty.
4. Inspect: check facts, logic, tone, sources, omissions and possible bias.
5. Test: ask whether the output works for different audiences, languages and accessibility needs.
6. Approve: ensure an accountable human authorises publication or decision use.
7. Record and improve: retain an appropriate audit trail and correct errors transparently.

AI-generated analysis should be labelled as preliminary when it has not been independently validated. Critical claims require primary-source verification. A leader should never cite an invented reference, accept a fabricated quotation or assume that a confident answer is a verified answer.

8. Bias, privacy, security and synthetic media

Bias can enter through historical training data, under-represented languages, proxy variables, problem framing or unequal access to communication channels. Fairness is not achieved merely by removing sensitive attributes; other variables may reproduce their effects. Leaders should test performance across relevant groups and examine false positives and false negatives.

Privacy requires data minimisation, purpose limitation, access control, retention rules and transparency. Communication analytics should collect only what is necessary. Employees should know what is being measured, why it is measured, who can access it and how long it is retained. Applicable data-protection, employment, records-management and intellectual-property requirements must be followed.

Security risks include prompt injection, data leakage, compromised accounts, malicious files and unauthorised reuse of confidential material. Organisational controls should include approved tools, identity management, secure storage, staff training and incident response.

Deepfakes are synthetic or manipulated audio, video or images that may imitate a person or event. They can damage reputations, manipulate employees or support fraud. Verification practices include checking the original source, confirming through a separate trusted channel, inspecting timing and context, and avoiding urgent action based on a single digital artefact. Leaders should establish a communication protocol for suspected manipulation.

Human oversight is especially important when communication affects employment, safety, reputation, access to services or legal rights. Oversight must be substantive: a human should understand the evidence, challenge the output and be able to reject it. Merely clicking “approve” is not meaningful governance.

9. Illustrative case: the Northbridge service redesign

Background. Northbridge, a fictional public-service organisation, introduced a new digital appointment process. Senior leaders sent weekly updates and measured email open rates. After six weeks, the open rate was 84%, so leaders assumed communication was successful. However, complaints increased and frontline staff reported inconsistent explanations.

Problem. The organisation had measured an output but not understanding, accessibility or operational impact. The analytics team proposed sentiment scoring of all employee comments and a ranking of departments by participation.

Analysis. A review found that the open-rate denominator excluded bounced messages and that several employees shared devices. A short anonymous survey showed that only 58% could correctly identify the new escalation route. Open comments revealed three themes: unclear exceptions, inadequate language support and uncertainty about who owned difficult cases. Interviews with service staff showed that the most connected supervisors were acting as informal brokers, but were overloaded. Automated sentiment classification mislabelled neutral comments containing the word “failure” because the word was used in a technical context.

Approach. The leadership team replaced the single success metric with a measurement plan. It used a short knowledge check, accessibility testing, issue-resolution time, staff confidence and selected interviews. AI was used to draft plain-language alternatives and cluster comments, but a bilingual review group checked translations and themes. Personal data was removed before analysis. The organisation published a correction, clarified the escalation process and held facilitated sessions with supervisors and staff.

Outcome. After four weeks, correct identification of the escalation route increased from 58% to 81%, while unresolved exceptions fell. The organisation did not claim that communication alone caused the improvement; operational training and process changes occurred at the same time. The network findings led to workload support for broker supervisors rather than public ranking.

Learning points. First, reach is not understanding. Second, automated sentiment is a screening aid, not a final judgement. Third, network centrality can indicate responsibility and burden simultaneously. Fourth, transparent correction can strengthen trust. Finally, measurement is valuable only when it changes a responsible leadership decision.

10. An evidence-based leadership communication plan

An integrated plan can be organised as follows:

1. Diagnose: identify the decision, audience, communication barrier and baseline evidence.
2. Set objectives: specify the desired knowledge, confidence, behaviour or relationship outcome.
3. Map stakeholders: consider authority, impact, information needs, accessibility and likely concerns.
4. Choose channels: match channel to message complexity, urgency, confidentiality and audience access.
5. Design the message: use clear structure, evidence, acknowledgement of uncertainty and a route for questions.
6. Use AI selectively: draft, translate or analyse only within approved boundaries; verify every consequential output.
7. Deliver and listen: provide two-way opportunities and ensure managers can explain and reinforce the message.
8. Measure: collect a small balanced set of output, outcome and operational indicators.
9. Review equity and risk: examine who is missing, misclassified, burdened or exposed.
10. Adapt and report: explain what was learned, what changed and what remains uncertain.

This plan treats analytics as a feedback system rather than a scorecard. It also recognises that trust is built through consistent behaviour, not through measurement claims alone.

Contemporary developments and future practice

Future communication practice will increasingly involve multimodal AI, real-time translation, synthetic presenters, automated meeting assistants and integrated organisational analytics. These tools may improve access and reduce administrative work. They may also blur authorship, increase information volume and make authenticity harder to judge.

Leaders therefore need a form of data literacy that includes measurement design, uncertainty, privacy and algorithmic accountability. They also need communication judgement: knowing when a human conversation is more appropriate than an automated message, when silence is harmful, and when evidence is too weak for a confident claim. The durable professional capability is not tool mastery alone. It is the ability to combine evidence, empathy, ethical reasoning and accountable action.

Chapter Summary

Communication analytics links leadership objectives to evidence about reach, understanding, perception, behaviour and organisational outcomes. Effective indicators have clear definitions, valid denominators and a direct connection to decisions. Surveys require attention to question design, response bias, reliability, validity and appropriate interpretation. Engagement measures should not confuse activity with meaningful participation. Sentiment analysis can identify patterns at scale but requires human review, contextual interpretation and error testing. Network analysis can reveal collaboration, silos and brokerage, but raises privacy and surveillance concerns.

Generative AI can assist drafting, analysis and accessibility when used within approved boundaries. Leaders must verify facts, protect confidential data, test for bias, address accessibility, detect synthetic media and maintain substantive human oversight. A strong communication plan uses a baseline, explicit objectives, mixed evidence, transparent limitations and an adaptation cycle. The purpose of analytics is not to produce more numbers; it is to support fairer, clearer and more effective leadership action.

Key Terms

Communication analytics: Systematic use of communication data to inform leadership decisions.

Communication objective: A defined intended change in knowledge, perception, behaviour or relationship.

Indicator: An observable measure used to assess progress.

Output: An immediate product or activity produced by communication.

Outcome: A change resulting from communication.

Impact: A wider, longer-term organisational consequence.

Baseline: Evidence describing conditions before an intervention.

Validity: The extent to which a measure represents the intended construct.

Reliability: The consistency of a measurement process.

Response rate: The proportion of eligible invitees who provide a usable response.

Sentiment analysis: Computational classification or interpretation of affective orientation in text.

Triangulation: Use of multiple evidence sources to improve interpretation.

Network analysis: Study of nodes and relationships represented as edges.

Centrality: A family of measures describing a node’s structural position.

Density: The proportion of possible relationships that are observed under defined assumptions.

Generative AI: AI that produces new content from learned patterns and prompts.

Data minimisation: Collection and retention of only necessary data.

Deepfake: Synthetic or manipulated media that may imitate real people or events.

Human oversight: Accountable human evaluation and authority over an AI-supported output or decision.

Algorithmic bias: Systematic and consequential distortion associated with data, design or deployment.

Review Questions

Short Answer Questions

1. Distinguish an output from an outcome and give one example of each.
2. Why can a high message open rate fail to demonstrate understanding?
3. Define sentiment analysis and identify one limitation.
4. What is the difference between a node and an edge in network analysis?
5. State two responsibilities of a human reviewer of AI-generated communication.

Long Answer Questions

1. Explain how a leader could design a measurement plan for a major organisational change. Include objectives, indicators, data sources and decision rules.
2. Discuss the reliability and validity challenges involved in using employee surveys to measure communication quality.
3. Explain how engagement analytics, sentiment analysis and network analysis can complement one another.
4. Evaluate the ethical risks of using AI to analyse employee communication data.
5. Describe a responsible workflow for using generative AI to draft and publish a leadership announcement.

Analytical / Application Questions

1. An organisation reports a 90% email open rate, but only 55% of employees can identify the required action. Analyse what may have happened and propose three improved indicators.
2. A network map identifies one supervisor as the most central person in an organisation. Explain at least two possible interpretations and recommend an ethical management response.
3. An AI system classifies employee comments as mostly negative, while interviews suggest that employees are concerned but constructive. Design an investigation to determine why the results differ and how leaders should respond.

Multiple Choice Questions with Answers

Question 1

Which measure most directly assesses whether employees understood a communication?

A. Number of messages sent

B. Email delivery rate

C. Accuracy on a knowledge check

D. Number of communication channels used

Correct Answer: C. Accuracy on a knowledge check

Question 2

A change from 50% to 60% represents what absolute change?

A. 5 percentage points

B. 10 percentage points

C. 20 percentage points

D. 60 percentage points

Correct Answer: B. 10 percentage points

Question 3

What is the main limitation of treating email open rate as a complete measure of engagement?

A. It cannot show whether a message was delivered

B. It proves that recipients changed their behaviour

C. It may show access without comprehension or action

D. It is always calculated from a biased sample

Correct Answer: C. It may show access without comprehension or action

Question 4

In communication network analysis, what does an edge normally represent?

A. A person or organisational unit

B. A relationship or interaction between nodes

C. The average survey score

D. An AI-generated prediction

Correct Answer: B. A relationship or interaction between nodes

Question 5

Why should sentiment-analysis results be reviewed by humans?

A. Automated systems cannot process any text

B. Human review removes the need for data protection

C. Context, sarcasm and domain language can cause misclassification

D. Sentiment has no relationship to communication

Correct Answer: C. Context, sarcasm and domain language can cause misclassification

Question 6

Which practice best reflects data minimisation?

A. Collecting all available employee messages indefinitely

B. Collecting only data necessary for a defined purpose

C. Publishing individual network positions publicly

D. Retaining raw data after the project has ended without review

Correct Answer: B. Collecting only data necessary for a defined purpose

Question 7

What is the most appropriate use of generative AI in a high-stakes leadership announcement?

A. Publish its first draft without checking it

B. Use it for a draft, then verify facts and approve the final message

C. Ask it to invent supporting evidence

D. Let it decide which employees should be disciplined

Correct Answer: B. Use it for a draft, then verify facts and approve the final message

Question 8

Which statement about a deepfake is correct?

A. It is always an unintentionally distorted recording

B. It is synthetic or manipulated media that may imitate real people or events

C. It is a type of survey response bias

D. It is a secure method of identity verification

Correct Answer: B. It is synthetic or manipulated media that may imitate real people or events

Question 9

If 180 of 600 eligible invitees complete a survey, what is the response rate?

A. 18%

B. 30%

C. 42%

D. 60%

Correct Answer: B. 30%

Question 10

Which condition most strengthens a claim that communication caused an observed outcome?

A. A high number of messages was sent

B. The outcome was measured only after the intervention with no baseline

C. Evidence includes temporal order and a credible comparison or alternative-explanation check

D. The leader believes the communication was persuasive

Correct Answer: C. Evidence includes temporal order and a credible comparison or alternative-explanation check