Incident Response, Threat Hunting and Forensic Analytics

Introduction and Context

Incident response, threat hunting and forensic analytics form an integrated discipline for managing security events that may affect the confidentiality, integrity or availability of information systems. Incident response is principally an operational and governance activity: it coordinates decisions before, during and after an incident. Threat hunting is a proactive analytical activity that searches for adversary behaviour that automated controls have not reliably detected. Forensic analytics applies disciplined methods to digital evidence so that investigators can reconstruct events, test competing explanations and communicate conclusions that are technically and legally defensible.

The three activities are interdependent. A response team may begin with a SIEM alert, an EDR detection, a user report or an anomaly discovered by a hunter. Triage establishes whether the event is benign, suspicious or a confirmed incident. Investigation then combines telemetry, endpoint artefacts, identity records, network evidence and contextual knowledge. The resulting findings guide containment, eradication and recovery, while lessons learned improve detection engineering and organisational resilience.

A mature programme does not equate a high volume of alerts with security effectiveness. It measures whether important attacks are detected promptly, whether evidence is preserved, whether business disruption is controlled and whether recurrence becomes less likely.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the lifecycle of incident response and the decisions associated with each phase.
2. Integrate SIEM, EDR, identity, network and cloud telemetry into an investigative timeline.
3. Map observed activity to the MITRE ATT&CK knowledge base without treating a mapping as proof of attribution.
4. Design hypothesis-driven threat-hunting enquiries.
5. Apply statistical and forensic-analytic techniques to distinguish signal from noise.
6. Make and document defensible decisions about severity, containment, evidence and recovery.
7. Evaluate ethical, legal, governance and operational constraints in investigations.

1. Foundations of Incident Response

1.1 Events, alerts and incidents

An event is an observable occurrence, such as a process creation, authentication or file modification. An alert is an event or correlation of events judged sufficiently suspicious to require attention. An incident is a security event, or collection of events, that compromises or threatens organisational assets, operations or policy. The distinction matters because indiscriminate declaration of incidents exhausts response capacity, while under-classification delays action.

A useful triage classification is:

Severity should not be determined by technical novelty alone. It should consider affected assets, data sensitivity, privilege, attacker access, propagation potential, operational impact and legal obligations. A compromised domain administrator account may be high severity even before evidence of data theft exists.

1.2 Lifecycle model

A practical lifecycle contains preparation, identification and triage, containment, eradication, recovery, and lessons learned. These phases are iterative rather than strictly linear. New evidence may require a return from recovery to containment or from lessons learned to preparation.

Preparation establishes policy, authority, roles, contacts, playbooks, logging, time synchronisation, evidence storage, communications procedures, backup strategy and exercises. Preparation also defines what actions responders may take without obtaining additional approval. A technically capable team can fail if it cannot legally isolate a server, access a cloud tenant or disclose a breach.

Identification and triage validate the signal, determine scope and assign priority. The analyst records the initial time, source, affected identity or asset, observed behaviour, confidence, business owner and immediate risks. Triage should preserve volatile evidence where appropriate, but it must not delay a necessary safety or containment action.

Containment limits harm and adversary freedom of movement. Short-term containment may isolate an endpoint, disable a token or block an address. Long-term containment may place systems in a controlled network segment or impose additional authentication requirements. Every containment action has a cost: isolation may interrupt production, and account disablement may affect legitimate operations. Decisions should therefore record the threat addressed, expected benefit, operational cost and approval authority.

Eradication removes the cause and persistence mechanism. It may involve deleting malicious services, removing unauthorised accounts, rotating credentials, patching exploited vulnerabilities and rebuilding systems. Eradication based only on a visible file is unsafe when the attacker may have established additional persistence.

Recovery restores services from trusted states, monitors for recurrence and verifies business functionality. A system is not recovered merely because it is online. Validation should include integrity checks, authentication tests, security-control operation and heightened monitoring.

Lessons learned examines technical and organisational causes. It should produce specific owners and deadlines, such as a new EDR rule, a revised privileged-access process or an improved log-retention decision. Blame-oriented reviews reduce reporting quality; evidence-oriented reviews improve resilience.

2. Evidence, Telemetry and Investigative Architecture

2.1 Telemetry sources

A SIEM aggregates and correlates records from multiple sources. Important sources include:

• EDR process trees, command lines, hashes, network connections and isolation status;
• identity-provider and directory logs, including authentication, privilege and token events;
• DNS, proxy, firewall, VPN and flow records;
• operating-system logs, scheduled tasks, services, registry or configuration changes;
• email security events and cloud audit trails;
• application, database and data-loss-prevention records; and
• vulnerability, asset-inventory and configuration-management data.

EDR generally provides rich endpoint context and response controls, whereas SIEM provides cross-source correlation, retention and organisational visibility. Neither is complete. A SIEM correlation rule may identify a suspicious login but lack process context; EDR may reveal execution but not the business sensitivity of the accessed database.

2.2 Normalisation and time

Forensic analysis depends on accurate temporal ordering. Systems should use a consistent time standard, normally UTC, and retain the original timestamp, timezone or offset, ingestion time and parsing status. Analysts must distinguish event time from collection time. A delayed log may arrive after a later event, and clock drift can produce an apparent but false sequence.

A normalised record commonly includes timestamp, source, subject, action, object, result, host, account, network address, event identifier and confidence. Normalisation improves correlation but may discard important raw fields. Consequently, raw records should be retained where feasible, with documented parsing and transformation rules.

2.3 Evidence integrity and chain of custody

Digital evidence must be acquired, stored and analysed in a manner that supports reproducibility. Investigators should document who collected it, when, from which source, using what tool and configuration, where it was stored, who accessed it and what transformations occurred. Cryptographic hashing helps demonstrate that a file or image has not changed.

For a cryptographic hash function, an input of arbitrary length produces a fixed-length digest. If H(x) denotes the hash of evidence item x, an integrity check compares the acquisition digest with a later digest. Equality supports the conclusion that the bytes are unchanged under the assumptions of the hash function and the correctness of the process; it does not prove that the evidence is authentic, complete or correctly interpreted.

The principle of minimisation is also important. Investigators should collect information relevant to the authorised purpose, restrict access, protect personal data and retain it only as long as justified by policy, law and investigative need.

3. ATT&CK Mapping and the Attack Narrative

MITRE ATT&CK is a knowledge base describing adversary tactics, techniques and, in some cases, sub-techniques and procedures. Tactics express the adversary’s objective, such as credential access or lateral movement. Techniques describe how an objective may be achieved, such as phishing, valid accounts or remote services.

ATT&CK mapping supports three activities:

1. Communication: a common vocabulary allows technical and managerial teams to discuss behaviour.
2. Detection engineering: mapped techniques can be linked to data sources, analytics and control gaps.
3. Coverage assessment: organisations can compare likely threats with observed visibility and response capability.

Mapping is not attribution. The same technique may be used by many threat groups, penetration testers, administrators or malware families. A responsible report distinguishes observation from inference:

• Observation: PowerShell launched from an Office process.
• Interpretation: behaviour is consistent with command and scripting activity.
• Hypothesis: a document may have initiated execution.
• Attribution: a named actor performed the intrusion.

The last claim requires substantially stronger evidence and should not be inferred merely from technique overlap. ATT&CK also should not become a checklist. A technique is useful only when tied to evidence, confidence, detection logic and investigative significance.

4. Hypothesis-Driven Threat Hunting

Threat hunting is a structured search for malicious or policy-violating activity that may have evaded existing detection. It differs from unbounded searching because it begins with a proposition that can be tested and falsified.

A hunting hypothesis has five elements:

1. Threat premise: a plausible adversary behaviour or exposure.
2. Expected observations: telemetry that should exist if the premise is true.
3. Search scope: systems, identities, time period and data sources.
4. Decision rule: evidence that supports, weakens or rejects the hypothesis.
5. Action: escalation, detection improvement, risk acceptance or additional collection.

For example: “If an attacker obtained a privileged token, unusual privileged logins will be followed by access from a new device, rare administrative commands or abnormal data access.” The hunter then defines the population, baseline and exclusions before querying data.

A hunt should begin with asset and identity context. Rare activity on a laboratory workstation is not equivalent to rare activity on a domain controller. Useful enrichments include ownership, business function, vulnerability status, maintenance windows, identity risk, known software and geographic information.

4.1 Statistical reasoning in hunting

The base-rate problem is central. Even an accurate detector can produce many false positives when malicious events are rare. Let P(D) be the prior probability of malicious activity, P(A|D) the probability of an alert when activity is malicious, and P(A|¬D) the probability of an alert when it is benign. Bayes’ theorem gives:

The formula assumes the probabilities refer to the same population and that the alert definition is stable. It calculates the probability of malicious activity given an alert, not the probability of an alert given malicious activity.

Worked example. Suppose 1% of relevant events are malicious, a detector identifies 90% of malicious events, and it falsely alerts on 5% of benign events. Then:

P(D|A) = (0.90 × 0.01) ÷ [(0.90 × 0.01) + (0.05 × 0.99)

The numerator is 0.009 and the denominator is 0.0585, so the posterior probability is approximately 0.1538, or 15.38%. The alert is important, but most alerts in this population are still benign. Enrichment and correlation are therefore essential.

Common detector metrics are:

Here, TP is true positive, FP false positive and FN false negative. Precision describes the proportion of positive alerts that are correct; recall describes the proportion of actual malicious cases detected. These measures require a defined reference set and can be misleading when the truth labels are incomplete. Accuracy is often inappropriate for highly imbalanced security data because a classifier can appear accurate by labelling almost everything benign.

Analysts may also use a standardised score to identify unusual behaviour:

Here, x is an observed value, μ is the baseline mean and σ is the standard deviation. The score is informative when the baseline is sufficiently stable and observations are reasonably comparable. Heavy-tailed or seasonal security data may violate normal-distribution assumptions; percentile thresholds, robust statistics or time-series models may then be preferable. A high z-score indicates rarity, not maliciousness.

5. Forensic Analytics and Defensible Decision-Making

5.1 Timeline reconstruction

Timeline analysis aligns events from multiple systems to answer: what happened, when, through which account or process, and with what consequence? A good timeline preserves uncertainty. If clock drift is estimated at ±2 minutes, the report should not claim second-level ordering between events within that interval.

Analysts should use a layered timeline:

• Access layer: authentication, VPN, remote desktop and token events;
• Execution layer: process creation, scripts, services and scheduled tasks;
• Discovery and movement layer: enumeration, remote connections and administrative actions;
• Collection and impact layer: archive creation, data access, deletion or encryption; and
• Response layer: isolation, credential reset, restoration and validation.

A timeline is an analytical model, not a raw dump. Each important conclusion should link to source records, collection methods and confidence.

5.2 Entity and relationship analysis

Many intrusions are better understood as relationships than isolated alerts. Entities include users, hosts, processes, files, domains, cloud resources and data repositories. Edges represent actions such as logged in to, spawned, connected to, modified or accessed. Graph analysis can reveal a rare path from a low-trust workstation to a sensitive server, but it remains dependent on logging completeness and correct entity resolution.

Analysts should avoid confusing correlation with causation. Two events may share an IP address because of NAT, a proxy or cloud infrastructure. A shared hash may indicate common legitimate software. Alternative explanations must be considered and recorded.

5.3 Confidence and competing hypotheses

A defensible investigation explicitly compares hypotheses. For example:

• H1: authorised administration;
• H2: compromised employee account;
• H3: malware-driven automation; and
• H4: logging or parsing error.

Evidence is assessed for how expected it would be under each hypothesis. Strong conclusions usually require converging evidence from independent sources, such as an identity event, endpoint process lineage and network connection. The absence of evidence is meaningful only when the relevant source was expected to collect and retain it. “No log found” is not equivalent to “the action did not occur.”

6. Operational Integration Across the Lifecycle

Incident response and hunting should share a feedback loop. A confirmed incident produces indicators, behaviours and control gaps. Hunters search for related activity across the environment; detection engineers convert validated patterns into durable analytics; system owners remediate exposure; and the response team tests whether the improvement works.

Useful operational measures include:

• mean time to detect (MTTD);
• mean time to acknowledge (MTTA);
• mean time to contain (MTTC);
• time from initial access to detection;
• percentage of critical assets with usable telemetry;
• proportion of high-severity incidents with preserved evidence; and
• recurrence rate after corrective action.

These metrics require careful definitions. For example, MTTD can be measured from the earliest evidenced malicious activity to the first reliable detection, but the earliest activity may be unknown. Organisations should therefore report measurement boundaries and confidence rather than presenting apparently precise but unsupported values.

Automation can enrich alerts, isolate hosts, revoke tokens and execute repeatable collection. However, high-impact actions should have approval controls, exception handling and audit logs. A machine-learning system may rank anomalies, but it should not be treated as an independent fact-finder. Model drift, biased training data, adversarial manipulation and opaque explanations can all impair investigative fairness and reliability.

7. Illustrative Case Study: Suspicious Cloud and Endpoint Activity

Background

A fictional financial-services organisation receives an EDR alert showing a scripting interpreter launched by a word-processing application on a finance employee’s laptop. Shortly afterwards, the identity provider records a successful login to a cloud administration portal from an unfamiliar device. The account has ordinary user privileges but is a member of a finance application group.

Problem

The organisation must determine whether this is a malicious document, a stolen session, authorised activity or a detection artefact. It must protect financial data without destroying evidence or unnecessarily interrupting payroll operations.

Analysis

The responder first preserves the EDR process tree, relevant memory or volatile artefacts where authorised, the document hash, identity-provider records and cloud audit logs. Times are normalised to UTC, and raw records are retained. The process tree shows the word processor spawning a scripting interpreter with an encoded command. The document arrived in an email from an external sender and had not been seen previously in the organisation.

The identity event is unusual for the account but not conclusive. The device fingerprint is new, the login occurs shortly after script execution, and cloud audit logs show attempts to enumerate finance resources. Network telemetry shows a connection to a newly registered domain. These independent observations support a compromise hypothesis more strongly than any single alert.

The team maps the observed behaviour to relevant ATT&CK techniques for malicious-file execution, command and scripting activity, valid-account use and cloud discovery. The mapping communicates behaviour; it does not identify an actor.

Approach and solution

The incident commander authorises short-term isolation of the laptop and revocation of active sessions for the affected account. The security team blocks the malicious domain, preserves the cloud audit trail, resets credentials after checking for other active sessions and searches for the document hash, domain and related process behaviour across endpoints. Hunting identifies two additional received documents but no evidence of execution.

Investigators compare the cloud-access hypothesis with authorised administration and logging-error hypotheses. The endpoint process lineage, email artefact, unfamiliar device and cloud enumeration form a coherent sequence. The team therefore treats the event as a confirmed compromise while qualifying the precise data-access conclusion: enumeration was observed, but successful exfiltration was not demonstrated.

The laptop is rebuilt from a trusted image after evidence collection. Finance access is restored only after credential rotation, endpoint validation and a review of group membership. Detection engineering adds a correlation rule linking external document delivery, scripting execution and unusual cloud login context.

Outcome and learning points

The organisation contained the incident before confirmed data exfiltration. It improved cloud logging, implemented stronger phishing-resistant authentication for privileged workflows and revised its evidence-preservation checklist. The case demonstrates that defensible response depends on converging evidence, proportionate containment, explicit uncertainty and a clear distinction between observed activity and inferred impact.

8. Governance, Ethics and Legal Considerations

Incident response can expose private communications, health information, employee records and customer data. Investigators must operate under an approved purpose, defined authority and least-privilege access. Monitoring employees without appropriate notice or legal basis may create privacy, labour and regulatory consequences. Cross-border cloud evidence may also involve jurisdictional restrictions and contractual obligations.

Organisations should maintain an incident-response policy, evidence-handling standard, retention schedule, breach-notification decision process and records of authority. Legal counsel, privacy officers, human resources, communications teams and regulators may need to be engaged according to the nature of the incident. Technical teams should not make unsupported legal conclusions; they should provide accurate facts, uncertainty and collection limitations.

Ethical practice includes avoiding unnecessary surveillance, protecting uninvolved individuals, reporting material uncertainty, not altering evidence to support a preferred narrative and ensuring that automated risk scores do not become unreviewed disciplinary decisions. Responsible disclosure and coordinated communication are preferable to concealment when vulnerabilities affect customers or other organisations.

9. Challenges and Limitations

Several conditions limit investigative confidence:

• incomplete or misconfigured logging;
• short retention periods and destroyed volatile evidence;
• clock drift and inconsistent time zones;
• encrypted traffic and privacy-preserving technologies;
• shared accounts, NAT and cloud multi-tenancy;
• attacker deletion, tampering and use of legitimate tools;
• alert fatigue and changing baselines;
• dependencies on external providers; and
• pressure to declare a simple cause before analysis is complete.

No framework eliminates uncertainty. The goal is a transparent, reproducible and proportionate decision process. Reports should state what is known, what is probable, what remains possible, what evidence is missing and what action is justified despite uncertainty.

10. Contemporary Developments and Best Practices

Modern environments require response across endpoints, identity providers, SaaS platforms, containers, operational technology and software-supply chains. Identity telemetry is increasingly central because attackers can operate through valid credentials without deploying conventional malware. Detection-as-code and infrastructure-as-code enable version control, testing and review of analytic rules and response playbooks.

Generative AI can assist with query construction, summarisation, malware triage and translation of technical findings for decision-makers. It can also hallucinate evidence, expose sensitive data, reproduce bias or encourage overconfident conclusions. AI outputs should therefore be treated as analyst aids, validated against source records and logged when used in consequential decisions.

Best practice is to:

1. define authority, roles, severity and communication paths before incidents;
2. maintain asset, identity and data inventories;
3. collect high-value telemetry with tested time synchronisation and retention;
4. preserve raw evidence and document transformations;
5. use hypotheses, baselines and explicit decision rules in hunting;
6. correlate independent sources rather than relying on one indicator;
7. map behaviours to ATT&CK while separating mapping from attribution;
8. choose containment proportionately and record business consequences;
9. validate eradication and recovery, not merely service availability; and
10. convert lessons learned into owned, measurable improvements.

Chapter Summary

Incident response is a governed lifecycle comprising preparation, triage, containment, eradication, recovery and lessons learned. Effective response integrates SIEM, EDR, identity, network, application and cloud telemetry while preserving raw evidence, time context and chain of custody. ATT&CK provides a useful behavioural vocabulary and supports detection coverage, but technique mapping is not attribution. Threat hunting is strongest when driven by explicit hypotheses, expected observations, baselines and decision rules. Statistical measures such as Bayes’ theorem, precision, recall and standardised scores help analysts reason about rarity and detector performance, but their assumptions and data quality must be stated. Forensic analytics requires timelines, relationship analysis, competing hypotheses and transparent confidence. Ethical and legal practice demands authorised collection, privacy protection, proportionality, reproducibility and honest communication of uncertainty. Mature programmes use incidents to improve telemetry, detections, controls and organisational resilience.

Key Terms

Incident: A security event or set of events that compromises or threatens information assets, operations or policy.

Alert: A notification generated when activity meets a suspicious or risk-based condition.

Triage: The initial validation, prioritisation and scoping of a security event.

Containment: Actions that limit an adversary’s access, movement or impact.

Eradication: Removal of malicious code, persistence, unauthorised access and exploited weaknesses.

Recovery: Restoration and validation of trustworthy services.

SIEM: A platform that aggregates, normalises and correlates security records.

EDR: Endpoint detection and response technology providing endpoint telemetry and control.

Telemetry: Observable data generated by systems, users, applications and security controls.

Threat hunting: Proactive, hypothesis-driven searching for malicious or policy-violating activity.

ATT&CK: A knowledge base of adversary tactics, techniques and procedures.

Chain of custody: Documentation of evidence collection, handling, access and transfer.

Hash: A fixed-length digest used to help verify data integrity.

Hypothesis: A testable proposition about a possible explanation or behaviour.

Precision: The proportion of positive detections that are true positives.

Recall: The proportion of actual positive cases detected.

Base rate: The prior frequency of a condition in a defined population.

False positive: A benign event incorrectly classified as suspicious or malicious.

Forensic timeline: A time-ordered analytical reconstruction of relevant events.

Defensible decision: A documented decision supported by evidence, authority, proportionality and stated limitations.

Review Questions

Short Answer Questions

1. Distinguish an event, an alert and an incident.
2. What is the purpose of containment, and why can it create operational risk?
3. Why does ATT&CK mapping not by itself establish attribution?
4. Identify three elements of a hypothesis-driven hunt.
5. Why should event time be distinguished from log ingestion time?

Long Answer Questions

1. Explain each phase of the incident-response lifecycle and describe how the phases can interact iteratively.
2. Discuss how SIEM, EDR, identity and cloud telemetry complement one another during an investigation.
3. Explain the role of chain of custody, hashing and raw-record retention in defensible digital investigations.
4. Analyse the ethical and governance issues that arise when investigators examine employee and customer data.
5. Evaluate the benefits and limitations of using AI to support incident response and forensic analytics.

Analytical / Application Questions

1. A detector has a 90% detection rate, a 5% false-alert rate and operates in a population where 1% of events are malicious. Calculate the probability that an alerted event is malicious and explain the operational implication.
2. Design a hypothesis-driven hunt for possible abuse of a privileged cloud account. Specify the hypothesis, data sources, expected observations, exclusions and escalation criteria.
3. Construct a defensible investigative plan for an endpoint alert followed by an unusual cloud login. Identify immediate actions, evidence to preserve, competing hypotheses, containment options and recovery validation steps.

Multiple Choice Questions with Answers

Question 1

Which activity is primarily associated with the preparation phase of incident response?

A. Deleting malicious persistence

B. Defining roles, authority, contacts and logging requirements

C. Rebuilding a compromised endpoint

D. Conducting post-incident lessons learned

Correct Answer: B. Defining roles, authority, contacts and logging requirements

Question 2

What is the principal purpose of triage?

A. To assign blame to the suspected user

B. To validate, prioritise and scope an event

C. To permanently remove all malware

D. To publish an attribution report

Correct Answer: B. To validate, prioritise and scope an event

Question 3

Why should ATT&CK technique mapping not be treated as attribution?

A. ATT&CK contains no technical behaviours

B. The same techniques can be used by different actors and legitimate administrators

C. ATT&CK applies only to physical attacks

D. Technique mapping does not require evidence

Correct Answer: B. The same techniques can be used by different actors and legitimate administrators

Question 4

In the chapter’s Bayesian example, what is approximately the probability that an alert represents malicious activity?

A. 1.00%

B. 5.00%

C. 15.38%

D. 90.00%

Correct Answer: C. 15.38%

Question 5

Which metric is defined as TP ÷ (TP + FP)?

A. Recall

B. Precision

C. Specificity

D. Base rate

Correct Answer: B. Precision

Question 6

What does a high z-score most directly indicate in a threat-hunting baseline?

A. Confirmed maliciousness

B. High-impact attribution

C. Rarity relative to the selected baseline

D. Successful data exfiltration

Correct Answer: C. Rarity relative to the selected baseline

Question 7

Which statement best describes a defensible forensic conclusion?

A. It reports only the investigator’s preferred explanation

B. It separates observations, inferences, uncertainty and limitations

C. It assumes missing logs prove that an event did not occur

D. It relies on a single indicator whenever possible

Correct Answer: B. It separates observations, inferences, uncertainty and limitations

Question 8

Which action is an example of short-term containment?

A. Isolating a suspected endpoint from the network

B. Writing the annual security strategy

C. Deleting all historical logs

D. Closing the incident without review

Correct Answer: A. Isolating a suspected endpoint from the network

Question 9

Why is accuracy often a poor primary measure for highly imbalanced security data?

A. It cannot be calculated from labelled data

B. A classifier can appear accurate by labelling nearly everything benign

C. It always equals recall

D. It measures only response time

Correct Answer: B. A classifier can appear accurate by labelling nearly everything benign

Question 10

Which practice best supports evidence integrity?

A. Analysing only transformed records and discarding originals

B. Allowing unrestricted access to collected data

C. Documenting collection, handling, access and cryptographic hashes

D. Changing timestamps to produce a convenient sequence

Correct Answer: C. Documenting collection, handling, access and cryptographic hashes