Ethical Hacking & Digital Forensics

Chapter 3
Security Architecture, Networking and Operating-System Fundamentals

Post Graduate Program in Information Technology
Ethical Hacking & Digital Forensics   •   Post Graduate

Security Architecture, Networking and Operating-System Fundamentals

Introduction and Context

Ethical hacking and digital forensics depend on an accurate mental model of how computing systems are designed, connected, operated and observed. An attack is not merely an isolated exploit: it is a sequence of actions across identities, protocols, hosts, applications, databases and trust boundaries. Similarly, forensic interpretation is not simply the recovery of files. It involves reconstructing events from artefacts generated by operating systems, network devices, applications, cloud services and users.

This chapter establishes the technical foundation for attack-path assessment and forensic analysis. It examines security architecture, TCP/IP networking, identity and access control, cryptography, Windows and Linux internals, applications, databases and cloud environments. The emphasis is analytical: a security professional must understand both intended system behaviour and the deviations that may indicate compromise.

Learning Objectives

By the end of this chapter, a learner should be able to:

  1. Explain how security architecture, trust boundaries and defence-in-depth influence attack paths.
  2. Analyse TCP/IP communication, common protocols and network evidence.
  3. Distinguish authentication, authorisation, accounting and identity federation.
  4. Explain the security purposes and limitations of cryptographic controls.
  5. Compare relevant Windows and Linux internals and their forensic artefacts.
  6. Assess application, database and cloud attack surfaces.
  7. Interpret logs and artefacts as evidence while accounting for time, provenance and uncertainty.
  8. Apply these concepts to an ethical hacking and digital-forensics scenario.

1. Security Architecture as an Analytical Model

Security architecture describes how assets, identities, processes, controls and trust relationships are arranged. It is broader than a firewall diagram. A useful analysis begins by identifying:

  • Assets: information, services, credentials, devices, workloads and business processes.
  • Actors: users, administrators, applications, services, attackers and automated agents.
  • Trust boundaries: points at which data or authority crosses between security domains.
  • Security controls: preventive, detective, corrective and compensating mechanisms.
  • Security objectives: confidentiality, integrity, availability, authenticity, accountability and privacy.

A conventional perimeter architecture places internal systems behind network controls. This remains useful for segmentation, but it is insufficient when users, applications and data operate across remote networks and cloud providers. Zero-trust architecture instead treats every access request as requiring explicit evaluation. It does not mean that trust is impossible; it means that trust is limited, contextual and continually assessed.

A practical access decision can be represented conceptually as:

Permit = Policy(subject, resource, action, context)

Here, subject is an identity or workload, resource is the requested object, action is the operation, and context may include device posture, location, time, authentication strength and risk. This is a policy model rather than a universal numerical formula. Its quality depends on accurate identity data, reliable telemetry and well-defined policy rules.

Defence in Depth and Attack Paths

Defence in depth uses independent or partially independent controls so that failure of one control does not expose the entire system. For example, a web application may be protected by secure coding, an identity provider, network segmentation, endpoint detection, database authorisation and monitored backups.

An attack path is a sequence of reachable states through which an actor may move from an initial condition to an objective. A simplified path may be:

Phishing message → stolen session token → cloud mailbox → privileged service account → sensitive database

Attack-path analysis asks whether each transition is technically possible, what privilege is required, which control should prevent or detect it, and what evidence would remain. The analysis must distinguish theoretical reachability from practical exploitability. A vulnerable service may be unreachable, patched, strongly isolated or protected by compensating controls.

Layered security architecture and trust boundaries UsersIdentity devices AccessMFA, policy, VPN NetworkSegmentation, IDS ApplicationsAPIs and services Data storesFiles and databases MonitoringLogs and response Trust boundaries must be identified, controlled and monitored
Layered security architecture showing users, network controls, applications, data and monitoring
Figure: Layered security architecture showing users, network controls, applications, data and monitoring

Figure: A layered architecture in which identity, network, application, data and monitoring controls interact across trust boundaries.

2. TCP/IP and Network Foundations

Layering and Encapsulation

The TCP/IP model is commonly described using application, transport, internet and link layers. Data is encapsulated as it moves downward through a sender's stack and decapsulated by the receiver. An application message becomes a transport segment or datagram, an IP packet, and a link-layer frame. Each layer provides services to the layer above and relies on assumptions made by the layer below.

LayerMain concernsExamples of evidence
ApplicationNaming, requests, data formatsHTTP requests, DNS queries, email headers
TransportPorts, reliability, multiplexingTCP flags, UDP flows, connection states
InternetAddressing and routingIP addresses, TTL, routes, fragmentation
LinkLocal delivery and media accessMAC addresses, ARP, switch records

An IP address identifies an interface or logical endpoint, not necessarily a person. Attribution therefore requires correlation with authentication logs, DHCP records, endpoint evidence, NAT translations and time sources.

TCP and UDP

TCP provides a connection-oriented byte stream. Its three-way handshake uses SYN, SYN-ACK and ACK to establish sequence-number state. TCP supports retransmission, flow control and congestion control, but it does not inherently provide confidentiality or proof of human identity. TLS is normally used above TCP to provide cryptographic protection.

UDP is connectionless and does not guarantee delivery, ordering or duplicate suppression. It is suitable for DNS, streaming, tunnelling and protocols where the application manages reliability. A UDP packet observed in a capture does not by itself prove that an application successfully received or processed it.

Ports identify transport-layer service endpoints. A listening port indicates that a process may accept traffic; it does not prove that the service is vulnerable. Service identification should combine port data with banners, protocol negotiation, configuration and process ownership.

Core Protocols and Their Security Significance

  • DNS: translates names to records. Attackers may abuse spoofing, poisoning, unauthorised transfers or covert channels. Investigators should preserve resolver logs and consider caching and TTL effects.
  • HTTP/HTTPS: carries web requests and responses. HTTPS protects content in transit when certificate validation and endpoint security are sound, but it does not make the application trustworthy.
  • SMTP, IMAP and POP: support email transport and retrieval. Headers, authentication events and mail-server logs can connect a message to infrastructure and accounts, although forwarding and spoofing complicate attribution.
  • DHCP: assigns network configuration. DHCP logs are important for mapping an address to a device at a particular time.
  • ARP: maps local IP addresses to link-layer addresses. ARP manipulation can redirect local traffic, but an ARP table is volatile evidence.
  • ICMP: supports diagnostics and error reporting. Blocking all ICMP can impair troubleshooting and path maximum-transmission-unit discovery; control should be risk-based.

Network Evidence and Time

Network evidence includes full packet captures, flow records, firewall logs, proxy records, DNS logs, VPN logs, wireless controller data and cloud network-flow records. Packet captures provide detail but may be absent, encrypted or too large. Flow records provide metadata such as source, destination, port, byte count and duration but usually cannot reveal payload content.

Time normalisation is essential. A timeline should record the timestamp, time zone or offset, clock source, precision and possible drift. Correlation should not assume that two events with identical displayed times occurred simultaneously. Where possible, investigators use authenticated time synchronisation and preserve original timestamps alongside normalised values.

3. Identity, Access Control and Cryptography

Identity and Access Control

Identity answers “who or what is requesting access?” Authentication verifies a claimed identity; authorisation determines permitted actions; accounting records activity. These are distinct functions. A successful login does not imply permission to read every file, and an authorisation record does not prove that the human owner personally performed the action.

Common access-control models include:

  • Discretionary access control (DAC): resource owners can grant permissions.
  • Mandatory access control (MAC): centrally enforced labels and clearances constrain access.
  • Role-based access control (RBAC): permissions are assigned to roles, which are assigned to subjects.
  • Attribute-based access control (ABAC): decisions use attributes of subjects, resources, actions and context.

Least privilege grants only the access required for an approved task. Privilege separation places sensitive operations in a distinct process or service. Separation of duties prevents one actor from completing a high-risk process without independent review.

Identity federation allows one authority to provide assertions to another service. SAML, OAuth 2.0 and OpenID Connect solve different problems: OAuth is an authorisation framework; OpenID Connect adds an identity layer; SAML commonly conveys authentication and attribute assertions in enterprise environments. Misconfigured redirect URIs, excessive token scope, weak token storage and inadequate revocation can create significant attack paths.

Cryptographic Foundations

Cryptography supports confidentiality, integrity, authentication and non-repudiation claims, but each claim depends on key management and protocol context.

  • Symmetric encryption uses the same secret key for encryption and decryption. It is efficient for bulk data but requires secure key distribution.
  • Asymmetric cryptography uses related public and private keys. It supports key exchange and digital signatures but is computationally more expensive.
  • Hash functions map arbitrary input to a fixed-length digest. A secure cryptographic hash should resist preimage, second-preimage and collision attacks under its intended threat model.
  • Message authentication codes (MACs) provide integrity and authentication to parties sharing a secret.
  • Digital signatures use a private key to sign and a public key to verify. They establish that the corresponding private key produced the signature, not necessarily which human operated it.

For password storage, systems should use a password-hashing or key-derivation function with a unique salt and an appropriate cost, rather than reversible encryption or a fast general-purpose hash. A salt prevents identical passwords from producing identical stored values and frustrates precomputed tables; it does not make a weak password strong.

Encryption in transit does not protect data after it reaches an authorised endpoint. Encryption at rest does not prevent misuse by a process that already has decryption privileges. Key rotation, access logging, hardware protection, backup handling and revocation are therefore as important as algorithm selection.

4. Windows and Linux Internals for Security Analysis

Windows

Windows security is organised around principals, security identifiers, access tokens, object security descriptors and policy enforcement. An access token represents a process or thread's security context, including user and group identifiers and privileges. A security descriptor contains ownership and access-control information for an object.

Important evidence sources include Windows Event Logs, registry hives, Prefetch data, Windows services, scheduled tasks, PowerShell operational logs, browser artefacts, link files, jump lists, volume shadow copies and the NTFS file system. The Security log may record logon, account-management and object-access events when appropriate auditing is enabled. Sysmon can add process creation, network connection, image-load and file-related telemetry, but its usefulness depends on configuration and retention.

The registry is a hierarchical configuration database containing system and user settings. It may reveal persistence mechanisms, connected devices, user activity and software configuration. Registry timestamps and last-write information must be interpreted cautiously: they do not automatically represent the time a person performed an action.

Linux

Linux separates user and group identities, file permissions, processes, capabilities and kernel-mediated operations. Traditional file permissions distinguish owner, group and others, with read, write and execute bits. Access-control lists and security frameworks such as SELinux or AppArmor can impose additional restrictions.

Key artefacts include systemd journal data, traditional authentication logs, shell histories, SSH configuration and keys, cron entries, process information under /proc, loaded modules, network sockets, package databases and file-system metadata. The presence of a command in shell history is useful but not conclusive: history may be disabled, edited, incomplete or generated by a different user context.

Linux process analysis should consider parent-child relationships, executable paths, environment variables, open files, network sockets, namespaces and capabilities. A process running as root is high risk, but a non-root process may still be dangerous if it possesses a powerful capability or accesses sensitive service credentials.

File Systems and Volatile Evidence

File metadata, journal records, deleted entries and slack space can support reconstruction, but recovery is not guaranteed. Solid-state-drive garbage collection and TRIM can remove recoverable blocks. Full-disk encryption may protect data when a device is powered off while leaving it accessible when unlocked.

Volatile evidence—memory contents, active connections, running processes, decrypted keys and mounted volumes—may disappear on shutdown. Acquisition decisions must balance volatility, legal authority, system availability and the risk of altering evidence. A forensic image should be acquired using validated tools, documented procedures and write protection where appropriate. Hash values can support integrity verification, but a matching hash proves equality of data, not the truth of an interpretation.

5. Applications and Databases

Applications expose attack surfaces through input handling, authentication, session management, file processing, APIs and dependencies. The major analytical questions are: what input is accepted, how it is interpreted, what identity is applied, what data is returned, and what is logged?

Common application weaknesses include injection, broken access control, insecure direct object references, cross-site scripting, server-side request forgery, unsafe deserialisation and insecure default configuration. Testing must be authorised, scoped and designed to avoid unnecessary data modification. A proof of concept should demonstrate risk with the least impact possible.

Databases enforce data storage, querying, transactions and access control. SQL injection occurs when untrusted input changes the intended structure of a query. Parameterised queries separate code from data and are the primary defensive control. Least-privileged database accounts, network restriction, secret management, auditing and encrypted backups provide additional protection.

A database transaction commonly follows atomicity, consistency, isolation and durability (ACID) principles. These properties concern transaction processing; they do not by themselves guarantee correct authorisation or accurate business logic. Forensic sources may include database audit logs, transaction logs, binary logs, query histories, application logs, replicas and backups. Investigators must understand retention, replication delay and whether logs can be altered by administrators.

6. Cloud Architecture and Shared Responsibility

Cloud systems combine virtual machines, containers, managed databases, object storage, serverless functions, identity services and software-defined networks. Their elasticity changes evidence collection: resources may be ephemeral, logs may be distributed across regions, and provider-controlled infrastructure may be inaccessible to the customer.

The shared-responsibility model assigns different duties to the provider and customer depending on the service model. The provider generally secures the underlying facilities and infrastructure; the customer remains responsible for configuration, identities, data and workloads to a degree that varies among infrastructure, platform and software services.

Cloud attack paths frequently arise from excessive permissions, public storage, exposed management interfaces, long-lived access keys, insecure instance metadata access, weak tenant isolation assumptions and unmonitored administrative APIs. Security analysis should inventory identities, roles, trust policies, security groups, routes, storage policies, key usage and control-plane activity.

Forensics in cloud environments requires early log preservation. Relevant sources include identity-provider events, cloud audit trails, virtual-network flow logs, workload logs, object-access records, container orchestration events and provider support records. Evidence acquisition must respect jurisdiction, contractual terms, privacy obligations and the provider's forensic procedures.

7. Applied Case: Investigating a Suspected Cloud-to-Database Intrusion

The following is an illustrative case designed for academic analysis.

Background

A financial-services organisation operates a public web application in a cloud environment. Customer data is held in a managed relational database. The organisation uses a central identity provider, a web application firewall, cloud audit logging and endpoint monitoring on an administration workstation.

Problem

The security team observes an unusual database export. An administrator's account was used from an unfamiliar network address, and an application server made an unexpected outbound connection shortly beforehand. The team must determine whether the event was caused by stolen credentials, application compromise, or an authorised administrative action.

Analysis

Investigators first preserve cloud audit logs, identity-provider records, database audit data, WAF records, endpoint telemetry and relevant configuration snapshots. They create a normalised timeline while retaining original timestamps and offsets.

The identity logs show a successful login without the organisation's usual second factor, followed by creation of a new access token. The application logs contain a request with an unusual parameter pattern and a response indicating an internal service interaction. Network-flow records show the application server contacting a cloud metadata endpoint and then making a database connection under a service identity.

The evidence suggests a chained path: an application weakness enabled access to temporary credentials; those credentials were used to query the database; the export was then written to temporary object storage. However, the conclusion must remain proportional to evidence. Flow records alone do not prove the contents of the export, and the identity record does not establish the attacker's physical identity.

Approach and Solution

The organisation revokes the exposed credentials, rotates relevant secrets, restricts metadata access, removes public object-storage permissions and narrows the service identity's database privileges. It deploys parameterised queries and adds detection for unusual metadata access, token creation and bulk database reads. A legally authorised forensic review of the application image and endpoint evidence is conducted, and affected data is assessed under applicable notification requirements.

Outcome

Correlation demonstrates that the database export was technically consistent with a compromised application identity rather than an administrator's workstation. Containment limits further access, while improved logging allows the organisation to distinguish control-plane activity from application-plane activity.

Learning Points

  1. Identity, application, network and database evidence must be correlated.
  2. Temporary credentials can be highly privileged even when no password is stolen.
  3. Cloud logs are essential evidence and should be configured before an incident.
  4. A plausible attack path is not automatically a proven conclusion.
  5. Remediation must address both the immediate credential and the architectural weakness.

8. Professional, Ethical and Governance Considerations

Ethical hacking requires explicit authorisation, a defined scope, rules of engagement, safety limits and a reporting process. Testing personal data, production systems or third-party infrastructure without authority may violate law, policy and professional ethics. Examiners should minimise collection, protect evidence, document every action and maintain chain of custody.

Digital evidence should be relevant, authentic, reliable and sufficiently complete for the question being asked. Analysts should record acquisition methods, tool versions, hash values, time assumptions, exclusions and alternative explanations. Confirmation bias is a serious risk: investigators must actively seek evidence that could disprove an initial hypothesis.

Privacy and data-protection principles require purpose limitation, data minimisation, access restriction and appropriate retention. Cloud investigations may cross borders and involve provider personnel, so legal process and contractual obligations must be considered. Automated and AI-assisted detection can prioritise events, but it may produce false positives, inherit biased training data or obscure reasoning. Human review remains necessary for high-impact decisions.

9. Contemporary Developments and Best Practices

Modern environments increasingly use identity-centric controls, software-defined segmentation, infrastructure as code, containers, serverless computing and extended detection and response. These developments improve scalability but increase dependency on configuration, APIs and central identity systems.

Effective practice includes:

  • maintain an asset and data-flow inventory;
  • enforce phishing-resistant multi-factor authentication for privileged access;
  • use least privilege, short-lived credentials and separation of duties;
  • segment management, application and data networks;
  • secure applications through threat modelling, code review and dependency management;
  • centralise tamper-resistant logs with tested retention;
  • synchronise time and document clock assumptions;
  • test backups and incident-response playbooks;
  • validate forensic tools and preserve original evidence;
  • measure control effectiveness through realistic, authorised exercises.

No control is absolute. Architecture must therefore be evaluated as a socio-technical system involving people, processes, technology, suppliers and governance.

Chapter Summary

Security architecture provides the structure for analysing assets, identities, trust boundaries and controls. TCP/IP knowledge allows professionals to interpret addressing, ports, protocols, encapsulation and network evidence without confusing technical endpoints with human attribution. Identity analysis separates authentication, authorisation and accounting, while access-control models explain how permissions should be assigned.

Cryptography protects confidentiality, integrity and authenticity only when algorithms, protocols and key management are correctly implemented. Windows and Linux expose different but complementary forensic artefacts, and volatile evidence requires careful acquisition decisions. Applications and databases introduce input, session, privilege and transaction risks. Cloud environments extend attack surfaces into identity systems, management APIs and provider-controlled evidence.

The central professional skill is correlation: constructing and testing an evidence-based attack path while recording uncertainty, legal constraints and alternative explanations.

Key Terms

Attack path: A sequence of reachable states leading from an initial condition to an objective.

Authentication: Verification of a claimed identity.

Authorisation: Determination of which actions an authenticated subject may perform.

ABAC: Access control based on subject, resource, action and contextual attributes.

ACID: Atomicity, consistency, isolation and durability properties of transactions.

Chain of custody: Documented control and handling history of evidence.

Cryptographic hash: Fixed-length digest produced by a cryptographic hash function.

Digital signature: Cryptographic evidence created with a private key and verified with a public key.

Defence in depth: Use of multiple complementary security controls.

Encapsulation: Wrapping data with control information as it passes through protocol layers.

Identity federation: Use of one authority's identity assertions by another service.

Least privilege: Granting only the access required for an authorised purpose.

MAC: Message authentication code based on a shared secret.

RBAC: Access control in which permissions are assigned to roles.

Trust boundary: A transition between domains with different security assumptions.

Volatile evidence: Evidence that can disappear or change when a system state changes.

Zero trust: Architecture based on explicit, contextual and continuously evaluated access decisions.

Review Questions

Short Answer Questions

  1. Why is an IP address insufficient by itself to identify a human user?
  2. Distinguish authentication from authorisation and accounting.
  3. What security properties can a digital signature provide?
  4. Name two volatile evidence sources in a live system.
  5. Why should cloud audit logs be preserved early in an investigation?

Long Answer Questions

  1. Explain how defence in depth and trust-boundary analysis support attack-path assessment.
  2. Compare TCP and UDP, including their implications for security monitoring and forensic interpretation.
  3. Compare DAC, MAC, RBAC and ABAC, giving an appropriate use context for each.
  4. Discuss the principal Windows and Linux artefacts used in an endpoint investigation.
  5. Explain the shared-responsibility model and its implications for cloud security and forensics.

Analytical / Application Questions

  1. A web server accesses a database using a service account with administrative privileges. Analyse the attack path, identify weaknesses and propose layered controls.
  2. A login appears in identity logs at 10:00 UTC, while endpoint logs show local time 10:04 and a network flow begins at 10:01. Explain how you would construct and qualify the timeline.
  3. A cloud object store contains sensitive data and has public-read access enabled. Develop an ethical investigation and remediation plan that addresses evidence preservation, containment, identity, configuration and governance.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes authentication from authorisation?

A. Authentication verifies identity; authorisation determines permitted actions.

B. Authentication encrypts data; authorisation compresses it.

C. Authentication records events; authorisation synchronises clocks.

D. Authentication assigns IP addresses; authorisation resolves domain names.

Correct Answer: A. Authentication verifies identity; authorisation determines permitted actions.

Question 2

Which TCP feature is associated with establishing connection state?

A. DNS resolution

B. The three-way handshake

C. ARP broadcasting

D. ICMP fragmentation

Correct Answer: B. The three-way handshake

Question 3

What is the principal purpose of a unique password salt?

A. To make passwords reversible

B. To eliminate the need for a password

C. To prevent identical passwords from producing identical stored values

D. To provide network confidentiality

Correct Answer: C. To prevent identical passwords from producing identical stored values

Question 4

Which access-control model assigns permissions to roles?

A. RBAC

B. DAC

C. MAC address filtering

D. NAT

Correct Answer: A. RBAC

Question 5

What does a digital signature primarily allow a verifier to establish?

A. That a message was compressed

B. That the corresponding private key produced the signature and the content was not altered

C. That the signer was physically present

D. That the network was unavailable

Correct Answer: B. That the corresponding private key produced the signature and the content was not altered

Question 6

Which item is generally volatile evidence?

A. A powered-off archival disk image

B. A printed policy document

C. Running-process and memory contents

D. A retained database backup

Correct Answer: C. Running-process and memory contents

Question 7

What is the primary defensive control against SQL injection?

A. Parameterised queries

B. Increasing screen brightness

C. Disabling time synchronisation

D. Using longer table names

Correct Answer: A. Parameterised queries

Question 8

Under the shared-responsibility model, which responsibility normally remains with the customer?

A. Physical security of the provider's buildings

B. Manufacture of provider hardware

C. Configuration of customer identities and permissions

D. Maintenance of the provider's cooling systems

Correct Answer: C. Configuration of customer identities and permissions

Question 9

Why must an analyst preserve original timestamps and time-zone information?

A. To make every event appear simultaneous

B. To support reliable correlation and expose clock assumptions

C. To remove all uncertainty from evidence

D. To replace the need for logs

Correct Answer: B. To support reliable correlation and expose clock assumptions

Question 10

Which statement about network flow records is most accurate?

A. They always contain complete application payloads.

B. They can provide metadata but usually do not reveal full payload content.

C. They prove the identity of the human who sent traffic.

D. They are equivalent to a forensic disk image.

Correct Answer: B. They can provide metadata but usually do not reveal full payload content.

Ethical Hacking & Digital Forensics
Chapter 3 of 15