Security Operations, Monitoring and Analytics

Introduction

Cyber security is not achieved solely by deploying firewalls, endpoint protection, identity controls, or secure applications. Organisations must also observe what occurs within their environments, recognise meaningful deviations from expected behaviour, investigate evidence, and communicate findings quickly enough to reduce harm. Security operations provides this continuous defensive capability.

Security operations combines people, processes, and technologies to identify, analyse, contain, and learn from security events. Monitoring supplies visibility; logs provide evidence; indicators and threat intelligence add context; security information and event management (SIEM) platforms support correlation and investigation; and analytics help distinguish normal variation from suspicious activity.

This chapter focuses on practical foundations for diploma-level security operations. It explains how events become alerts, how analysts triage alerts, how baselines support anomaly detection, and how technical findings are converted into actionable communication.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the purpose and lifecycle of security operations.
2. Distinguish events, logs, alerts, indicators, incidents, and findings.
3. Describe log sources, collection requirements, normalisation, retention, and integrity.
4. Explain SIEM architecture, correlation, detection rules, and alert prioritisation.
5. Use threat intelligence and indicators of compromise responsibly.
6. Construct and interpret simple behavioural baselines and anomaly scores.
7. Triage alerts using evidence, severity, confidence, and business context.
8. Communicate security findings in a concise, actionable format.
9. Recognise operational, ethical, privacy, and governance limitations.

1. The Security Operations Function

A security operations function is responsible for maintaining defensive awareness across an organisation’s digital environment. Its activities commonly include monitoring, detection engineering, alert triage, incident response, vulnerability coordination, threat intelligence, reporting, and continuous improvement.

A useful operational lifecycle is:

Prepare → Collect → Detect → Triage → Investigate → Contain → Recover → Learn

Preparation defines assets, responsibilities, escalation paths, logging requirements, and playbooks. Collection gathers security-relevant telemetry. Detection identifies potentially harmful behaviour. Triage determines urgency and credibility. Investigation establishes what happened, when, how, and with what impact. Containment limits damage, recovery restores reliable service, and learning improves controls and procedures.

Security operations should not be understood as a purely technical activity. A failed login on a public web server, for example, may be routine internet noise. The same pattern against a privileged administrator account immediately after a new login from an unusual country may require urgent investigation. Business criticality, identity privilege, timing, and corroborating evidence affect the decision.

Security operations roles

Roles vary by organisation, but the following division is common:

Clear responsibility is essential. A SIEM can generate an alert, but it cannot by itself determine whether an employee’s unusual access is authorised, whether a server can be isolated safely, or whether a regulator must be notified.

2. From Events to Actionable Findings

Security vocabulary must be precise because imprecise language causes poor decisions.

• Event: An observable occurrence, such as a login, process creation, DNS query, or file modification.
• Log record: A stored representation of an event, usually containing time, source, action, result, and contextual fields.
• Indicator: A value or pattern associated with activity of interest, such as an IP address, domain, file hash, command pattern, or behavioural characteristic.
• Alert: A notification produced when a rule, model, or analyst identifies potentially significant activity.
• Finding: An assessed observation that explains evidence, significance, and recommended action.
• Incident: An event or set of events that has been confirmed or reasonably suspected to compromise confidentiality, integrity, availability, or policy compliance.
• Case: The investigation record containing alerts, evidence, decisions, communications, and actions.

An event is not automatically an incident. A single failed login is an event. Hundreds of failed logins followed by a successful login and unusual mailbox access may constitute an incident or a high-priority investigation.

Log quality dimensions

Useful logs should be:

• Accurate: They represent the underlying activity.
• Complete: Important fields and relevant sources are present.
• Timely: Records arrive quickly enough to support response.
• Consistent: Time zones, field names, and formats are understood.
• Available: Analysts can query them during an investigation.
• Protected: Unauthorised alteration and disclosure are prevented.
• Relevant: Collection has a defensible security or operational purpose.

Time synchronisation is especially important. If an endpoint clock is several minutes wrong, the apparent order of authentication, process execution, and network connection may be misleading. Network Time Protocol (NTP), documented time zones, and retention of original timestamps help maintain a reliable timeline.

3. Telemetry and Log Sources

A monitoring programme should begin with questions rather than products: Which assets are critical? Which actions would indicate misuse? What evidence is needed to confirm or reject those hypotheses? What response decisions must the evidence support?

Common sources include:

Collection should be risk-based. Logging every possible field can create excessive cost, storage, privacy exposure, and analyst overload. Conversely, collecting only firewall blocks may leave no evidence of activity occurring through permitted channels.

Normalisation and enrichment

Normalisation converts different vendor formats into a common analytical structure. For example, a source-specific field such as src_ip, clientAddress, or sourceAddress may be mapped to source.ip. Enrichment adds context such as asset owner, geographic information, identity privilege, vulnerability status, or whether an address belongs to a known cloud provider.

Normalisation supports consistent rules, while enrichment supports judgement. Neither eliminates the need to preserve original records. Original data may be required for forensic verification, legal review, or troubleshooting parser errors.

Retention and protection

Retention should reflect legal obligations, organisational policy, investigation needs, storage cost, and the expected dwell time of threats. Access to logs should follow least privilege. Sensitive fields, including personal data, credentials, tokens, and message content, require special protection. Logs should be transmitted and stored securely, and important administrative actions should themselves be logged.

4. SIEM Concepts and Architecture

A SIEM collects, processes, stores, searches, and correlates security-relevant data. A simplified flow is:

Figure: A simplified SIEM flow from telemetry collection to detection, investigation, and response.

A SIEM commonly performs four analytical activities:

1. Filtering: Removing irrelevant records or selecting records of interest.
2. Correlation: Relating events by identity, host, IP address, session, time, or other attributes.
3. Detection: Applying rules, thresholds, statistical methods, or threat intelligence.
4. Presentation: Displaying alerts, timelines, dashboards, and cases for human decisions.

Correlation can reduce isolated-event ambiguity. For example, a new administrator account, a privileged login from an unfamiliar device, and a large export from a sensitive database may be more significant together than separately.

Detection rules

A detection rule should identify a behaviour, explain its rationale, define required data, and specify an action. A good rule includes a name, logic, severity, confidence, affected assets, suppression or grouping conditions, response guidance, and test evidence.

Rules can be:

• Signature-based: Match known hashes, domains, commands, or patterns.
• Threshold-based: Trigger when a count or rate exceeds a limit.
• Sequence-based: Detect an ordered chain of events.
• Context-based: Combine activity with asset, identity, or business information.
• Behavioural: Detect deviation from an established pattern.

Rules require maintenance. Software changes, new cloud services, acquisitions, seasonal activity, and adversary adaptation can make a previously useful rule noisy or ineffective.

5. Threat Intelligence and Indicators

Threat intelligence is processed information about threats that supports a decision. Raw data becomes intelligence when it is evaluated, given context, and connected to an operational purpose.

Intelligence may be:

• Strategic: Trends, campaigns, risks, and implications for leadership.
• Operational: Adversary objectives, methods, and likely campaigns.
• Tactical: Techniques and procedures used by attackers.
• Technical: Observable indicators such as hashes, domains, IP addresses, and email artefacts.

An indicator is not proof of maliciousness. An IP address may be shared by many customers, a domain may be compromised temporarily, and a file hash may identify a legitimate tool used maliciously. Analysts should consider source reliability, indicator age, collection method, confidence, and local context.

Indicator handling should include:

1. Record the indicator and its type.
2. Record source, discovery time, confidence, and expiration date.
3. Search historical telemetry where feasible.
4. Check whether the indicator is shared infrastructure or a known false positive.
5. Correlate with identity, endpoint, network, and business context.
6. Apply blocking or alerting only when the operational risk is understood.

Threat intelligence platforms and standardised exchange formats can improve sharing, but automation should not turn unverified external data directly into destructive actions.

6. Baselines and Anomaly Analytics

A baseline describes expected behaviour over a defined period. Examples include normal login times for a service account, typical outbound data volume from a server, common destinations for a business application, or normal administrative activity for a user.

Baselines must account for seasonality, shifts, maintenance windows, business cycles, and legitimate change. A single average can be misleading. A payroll server may have different behaviour during monthly processing, and a university account may have different patterns during examinations.

For observations x₁, x₂, …, x_n, the arithmetic mean is:

Here, n is the number of observations and x̄ has the same units as each xᵢ. The mean is appropriate when observations are reasonably comparable and extreme values do not dominate interpretation.

A sample standard deviation is:

The denominator n − 1 is used when the observations are treated as a sample from a broader process. Standard deviation measures dispersion in the same units as the observations. It does not, by itself, establish that an observation is malicious.

A simple standardised score is:

where x is the current observation, x̄ is the baseline mean, and s is the sample standard deviation. The score is dimensionless. Under approximately stable, normally distributed conditions, a large absolute score may indicate unusual behaviour. This assumption often fails for security data, which can be skewed, seasonal, sparse, or affected by attacks. Therefore, z-scores should support—not replace—analyst judgement.

Worked example: outbound data volume

Suppose a server’s daily outbound volume over five comparable working days is 10, 12, 11, 9, and 8 GB.

The mean is:

The squared deviations are 0, 4, 1, 1, and 4, giving a total of 10 GB². The sample standard deviation is:

If the current volume is 14 GB:

This is unusual relative to the small baseline, but it is not automatically exfiltration. The analyst should examine destination, encryption, user or process identity, scheduled backups, data classification, and the server’s business role. A five-day baseline is also weak; more comparable observations would provide greater confidence.

Other useful methods include medians and percentiles for skewed data, rolling windows for changing behaviour, peer-group comparison, and exponentially weighted moving averages. More complex machine-learning models may identify relationships that fixed rules miss, but they introduce explainability, training-data, drift, and false-positive risks.

7. Alert Triage and Investigation

Triage is the disciplined process of deciding what an alert means, how urgent it is, and what should happen next. It is not merely sorting alerts by a vendor-generated severity label.

A practical triage sequence is:

1. Validate the alert: Confirm that the rule fired correctly and identify missing or malformed data.
2. Identify the entity: Determine the user, device, application, account, IP address, or cloud resource involved.
3. Assess context: Check privilege, asset criticality, vulnerability state, location, time, and maintenance activity.
4. Seek corroboration: Search related authentication, endpoint, DNS, network, email, and application records.
5. Establish a timeline: Identify the first and last relevant events and any sequence of escalation.
6. Assess scope: Search for other affected systems, accounts, indicators, or similar activity.
7. Assign priority: Consider impact, likelihood, confidence, and time sensitivity.
8. Choose an action: Close as benign, monitor, request validation, escalate, or initiate response.
9. Document the reasoning: Record evidence, uncertainty, decisions, and next steps.

A simple risk framing is:

This is a conceptual prioritisation model, not a universal measurement formula. Likelihood may reflect evidence and confidence; impact may reflect confidentiality, integrity, availability, safety, legal, and financial consequences. Organisations should define their own scales rather than treating the expression as a precise probability calculation.

Severity and confidence

Severity asks, “How harmful could this be?” Confidence asks, “How strong is the evidence that the activity is what we think it is?” A high-severity, low-confidence alert may require rapid validation. A medium-severity, high-confidence event affecting a critical system may also deserve immediate action.

Analysts should avoid two common errors:

• Premature closure: Dismissing unusual activity because one explanation appears plausible.
• Premature escalation: Declaring compromise without checking authorised change, shared infrastructure, or data quality.

8. Communicating Actionable Findings

A security finding should allow a recipient to understand the situation and make a decision. Technical detail is valuable only when it supports action.

A concise finding normally contains:

• Title: A specific statement of the issue.
• Executive significance: Why the organisation should care.
• Observed evidence: Relevant times, systems, accounts, indicators, and records.
• Assessment: What is known, suspected, and still uncertain.
• Scope: Affected or potentially affected assets.
• Severity and confidence: With a clear rationale.
• Recommended actions: Immediate containment and longer-term remediation.
• Owner and deadline: Who must act and by when.
• References: Case number, queries, evidence locations, and related intelligence.

For example: “At 14:22 UTC, account A accessed the finance database from an unmanaged device and exported 2.1 GB after a successful MFA reset. The account is privileged, the device is not registered, and no approved change ticket was found. Confidence is medium because export logging is incomplete. Validate the user immediately, suspend the session if unauthorised, preserve endpoint and database evidence, and review other access by the account during the preceding 24 hours.”

This is stronger than “Suspicious finance activity detected” because it describes evidence, uncertainty, and action.

9. Applied Case: Illustrative Ransomware Detection

Background. A regional logistics company operates Windows endpoints, cloud email, a virtual private network, and warehouse management systems. Its SIEM receives identity, endpoint, DNS, firewall, and cloud audit logs.

Problem. The monitoring team receives three alerts within ten minutes: a user clicked a newly registered domain, PowerShell executed on the user’s laptop, and the laptop made connections to an unusual external address. The alerts are initially separate and medium severity.

Analysis. A Tier 1 analyst validates that the email was delivered to the user and that the PowerShell event contains an encoded command. The analyst checks the user’s recent authentication and discovers a login from a new device shortly after the click. Endpoint telemetry shows a suspicious process creating files in a temporary directory. DNS data identifies connections from two additional laptops to the same domain. The domain is present in a threat-intelligence feed, but the feed is only moderately reliable and does not prove compromise.

The analyst correlates the events by user, device, domain, and time. The sequence—phishing delivery, execution, external communication, and spread to peer devices—increases confidence that the activity is malicious. The warehouse management server is not yet observed communicating with the domain, but it is business-critical and receives data from affected endpoints.

Approach and solution. The analyst escalates the case to incident response. Responders isolate the three laptops using endpoint controls, disable the affected user session, preserve volatile and persistent evidence where feasible, and block the domain after confirming that it is not required for business operations. Identity administrators reset credentials and review MFA changes. Network teams search for the domain, address, and related command patterns across historical logs. System owners verify backups and monitor the warehouse management environment.

Outcome. Investigation confirms that one laptop downloaded a credential-stealing payload. The other two devices contained the email but did not execute the attachment. No evidence shows access to the warehouse management server. The organisation restores the compromised laptop from a trusted image, removes the malicious mailbox message, and updates email and endpoint detections.

Learning points. Correlation converted low-confidence alerts into a meaningful sequence. Threat intelligence supported, but did not replace, local evidence. Asset criticality guided containment priorities. The case also revealed gaps: incomplete PowerShell logging on older endpoints, inconsistent device identity data, and no tested playbook for suspected credential theft.

10. Governance, Ethics, and Limitations

Monitoring can affect employees, customers, and other individuals. Organisations should define a legitimate purpose, collect proportionate data, restrict access, document retention, and communicate monitoring practices where required. Analysts must not browse personal content merely because it is technically accessible.

Important governance controls include role-based access, separation of duties, audit trails, evidence-handling procedures, documented escalation, legal review, and tested business continuity plans. Privacy regulations and sector-specific rules may govern personal data, cross-border transfers, breach notification, and retention. Exact requirements depend on jurisdiction and should be confirmed with legal and compliance specialists.

Automation and AI can assist with event summarisation, query generation, clustering, prioritisation, and playbook execution. However, automated systems may reproduce biased training data, hallucinate explanations, miss novel attacks, or take unsafe actions. Human approval should generally be required for disruptive actions such as disabling a large number of accounts, blocking shared infrastructure, or isolating critical production systems.

Operational limitations include incomplete telemetry, clock errors, encrypted traffic, shared cloud infrastructure, alert fatigue, changing baselines, vendor lock-in, and adversaries who deliberately imitate normal behaviour. Effective security operations therefore combines technical controls with documentation, testing, analyst development, and feedback from incidents.

11. Best Practices for a Diploma-Level Security Operations Programme

1. Begin with critical assets, important identities, and realistic attack scenarios.
2. Define required fields and time synchronisation before writing detection rules.
3. Preserve original records while providing normalised analytical fields.
4. Enrich alerts with asset ownership, privilege, vulnerability, and business context.
5. Use detection rules that have an owner, rationale, test cases, and review date.
6. Measure false positives and missed detections, not merely alert volume.
7. Maintain triage and containment playbooks and exercise them periodically.
8. Treat indicators as context-dependent evidence rather than automatic proof.
9. Use baselines that account for peer groups, seasonality, and legitimate change.
10. Record uncertainty explicitly and communicate recommended actions clearly.
11. Protect logs and cases as sensitive information.
12. Review incidents to improve collection, detections, training, and governance.

Chapter Summary

Security operations provides continuous defensive awareness through preparation, telemetry collection, detection, triage, investigation, response, and learning. Events become useful evidence when logs are accurate, timely, complete, protected, and placed in context. SIEM platforms normalise and correlate data from identity, endpoint, network, cloud, application, and other sources, but analysts remain responsible for judgement.

Threat intelligence supplies contextual information and indicators, although no indicator alone proves malicious activity. Baselines and anomaly analytics help identify deviations from expected behaviour; statistical measures such as the mean, sample standard deviation, and z-score are useful only when their assumptions and limitations are understood. Triage should assess evidence, confidence, impact, asset criticality, and scope. Findings should communicate what happened, what is uncertain, and what action is required.

Effective operations also depend on privacy, governance, evidence protection, tested playbooks, responsible automation, and continuous improvement. The central professional skill is converting diverse technical signals into reliable, proportionate, and actionable security decisions.

Key Terms

Event: An observable occurrence in a system or environment.

Log record: A stored representation of an event.

Indicator: A value or pattern associated with activity of interest.

Alert: A notification generated by a rule, model, or analyst.

Finding: An assessed observation containing evidence and significance.

Incident: Confirmed or suspected activity affecting security or policy.

SIEM: A platform that collects, processes, stores, searches, and correlates security data.

Normalisation: Mapping varied data formats into a consistent analytical structure.

Enrichment: Adding context such as asset ownership, privilege, or vulnerability information.

Correlation: Relating events using shared attributes or a time sequence.

Baseline: A description of expected behaviour over a defined period.

Anomaly: Behaviour that differs materially from an established expectation.

Threat intelligence: Processed threat information that supports a decision.

Triage: Initial assessment of alert validity, priority, scope, and response.

Severity: The potential consequence or harm associated with activity.

Confidence: The strength of evidence supporting an assessment.

Playbook: A documented procedure for handling a recurring alert or incident.

False positive: An alert incorrectly classified as suspicious or harmful.

Telemetry: Data produced by systems that supports monitoring and analysis.

Dwell time: The period an attacker remains active before detection or removal.

Review Questions

Short Answer Questions

1. What is the difference between an event, an alert, and an incident?
2. Why is time synchronisation important in security investigations?
3. Name four types of data that a SIEM may collect.
4. Why should an indicator not automatically be treated as proof of compromise?
5. What information should an actionable security finding contain?

Long Answer Questions

1. Explain the security operations lifecycle from preparation through learning, including the responsibilities of analysts and asset owners.
2. Describe how a SIEM collects, normalises, enriches, correlates, and presents security data.
3. Explain the purpose of behavioural baselines and discuss the limitations of using z-scores for security data.
4. Compare signature-based, threshold-based, sequence-based, context-based, and behavioural detections.
5. Discuss the ethical, privacy, governance, and operational risks associated with extensive security monitoring.

Analytical / Application Questions

1. A privileged account logs in from a new country, changes its MFA method, and downloads data from a sensitive system. Design a triage plan, identify additional evidence to collect, and recommend immediate actions.
2. A server normally transfers 8–12 GB per day but transfers 45 GB overnight. Explain how you would determine whether this is a backup, legitimate operational change, or possible exfiltration.
3. A SIEM rule produces 500 alerts per day, of which analysts close 480 as benign. Propose a structured method for improving the rule without creating a blind spot for genuine attacks.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes an alert from an incident?

A. An alert is always confirmed malicious activity, while an incident is never confirmed.

B. An alert is a notification of potentially significant activity, while an incident is confirmed or reasonably suspected harmful activity.

C. An alert is a stored log record, while an incident is a network connection.

D. An alert is produced only by a human, while an incident is produced only by a SIEM.

Correct Answer: B. An alert is a notification of potentially significant activity, while an incident is confirmed or reasonably suspected harmful activity.

Question 2

Which SIEM activity maps different vendor field names into a common structure?

A. Normalisation

B. Containment

C. Eradication

D. Escalation

Correct Answer: A. Normalisation

Question 3

Why should an analyst add asset ownership and privilege information to an alert?

A. To delete the original log record

B. To provide business and risk context for prioritisation

C. To guarantee that the alert is malicious

D. To prevent all future alerts

Correct Answer: B. To provide business and risk context for prioritisation

Question 4

In the baseline example with mean 10 GB and sample standard deviation approximately 1.58 GB, what is the approximate z-score for 14 GB?

A. 0.40

B. 1.58

C. 2.53

D. 14.00

Correct Answer: C. 2.53

Question 5

What is the most appropriate interpretation of a large absolute z-score in security monitoring?

A. It proves that malware is present.

B. It indicates unusual behaviour that requires contextual investigation.

C. It proves that the baseline is incorrect.

D. It should always trigger system shutdown.

Correct Answer: B. It indicates unusual behaviour that requires contextual investigation.

Question 6

Which threat-intelligence statement is most accurate?

A. Every listed IP address is permanently malicious.

B. An indicator should be assessed for source reliability, age, confidence, and local context.

C. Threat intelligence eliminates the need for local logs.

D. External indicators should always trigger automatic blocking.

Correct Answer: B. An indicator should be assessed for source reliability, age, confidence, and local context.

Question 7

During alert triage, what should generally occur before disruptive containment?

A. Delete all related logs

B. Validate the alert and assess context and scope

C. Publish the analyst’s hypothesis publicly

D. Disable every account on the network

Correct Answer: B. Validate the alert and assess context and scope

Question 8

Which log characteristic means that records arrive quickly enough to support response?

A. Timeliness

B. Completeness

C. Confidentiality

D. Normality

Correct Answer: A. Timeliness

Question 9

Which is the strongest example of actionable communication?

A. Suspicious activity detected.

B. The system may be compromised somehow.

C. At 14:22 UTC, a privileged account accessed a sensitive database from an unmanaged device after an MFA reset; validate the user and preserve evidence.

D. The SIEM generated an alert with severity medium.

Correct Answer: C. At 14:22 UTC, a privileged account accessed a sensitive database from an unmanaged device after an MFA reset; validate the user and preserve evidence.

Question 10

Why should human approval often be retained for disruptive automated actions?

A. Automation can never process logs.

B. Automated systems may lack context and can cause harmful false-positive actions.

C. Human approval makes indicators permanently valid.

D. Disruptive actions do not affect business services.

Correct Answer: B. Automated systems may lack context and can cause harmful false-positive actions.