Foundations of Software Engineering

Introduction

Software is now embedded in nearly every major activity: communication, banking, transport, healthcare, manufacturing, education and public administration. Yet software is not simply a collection of program statements. It is a socio-technical product involving requirements, architecture, source code, data, interfaces, documentation, infrastructure, users and organisational processes.

Software engineering is the disciplined application of engineering principles to the specification, development, operation, maintenance and evolution of software systems. It aims to produce software that is useful, dependable, maintainable and economically sustainable. A diploma-level software engineer must therefore understand both technical construction and the human, organisational and ethical context in which software is created and used.

This chapter establishes the vocabulary and conceptual foundations needed for later study of requirements engineering, design, programming, testing, project management, configuration management and software maintenance.

Learning Objectives

By the end of this chapter, you should be able to:

• define software engineering and distinguish it from programming;
• describe the major characteristics of software;
• classify common types of software and systems;
• identify stakeholders and explain their responsibilities;
• apply fundamental software engineering principles;
• explain important software quality attributes;
• analyse factors that contribute to project success or failure; and
• recognise professional, ethical, security and governance responsibilities.

1. What Is Software Engineering?

The term software includes executable programs, configuration data, databases, libraries, scripts and documentation needed to operate and maintain a system. Software engineering is the systematic, disciplined and measurable approach to developing, operating and maintaining software, and to studying the methods used for these activities.

The definition has several important implications:

1. Systematic: work follows an organised process rather than relying entirely on improvisation.
2. Disciplined: agreed practices, standards and reviews are used to control risk.
3. Measurable: quality, progress, defects, cost and performance can be assessed using suitable evidence.
4. Lifecycle-oriented: engineering continues after initial delivery through operation, support, adaptation and retirement.
5. Socio-technical: software interacts with people, organisations, hardware, networks, laws and business processes.

Programming is the activity of expressing an algorithm or solution in a programming language. It is essential, but it is only one part of software engineering. Engineering also involves understanding needs, managing uncertainty, designing an appropriate architecture, evaluating quality, documenting decisions, coordinating people and controlling changes.

Software engineering versus computer science and information technology

Computer science studies computational concepts, algorithms, data structures, programming languages and theoretical foundations. Software engineering applies relevant scientific and engineering knowledge to build dependable software products and services. Information technology focuses more broadly on using, operating and supporting computing infrastructure and information systems. These fields overlap, but their immediate questions differ:

A professional project may require knowledge from all three fields.

2. Characteristics of Software

Software differs from physical manufactured products in ways that strongly influence engineering practice.

2.1 Software is intangible

A program cannot be handled like a machine component. Its structure must be represented through models, code, tests, documentation and running behaviour. Because progress is less visible, teams need explicit work products and measurable acceptance criteria.

2.2 Software is developed, not manufactured

The main cost of software is usually concentrated in analysis, design, implementation, testing, deployment and maintenance. Once a digital product has been created, making another copy normally costs very little. Software does not wear out through physical friction, but it can become obsolete, insecure or unsuitable when its environment changes.

2.3 Software is highly changeable

Business rules, regulations, devices, operating systems, user expectations and threats change continuously. Change is therefore normal rather than exceptional. However, uncontrolled change creates defects, inconsistent documentation, technical debt and schedule disruption.

2.4 Software complexity is often essential

A bridge usually follows physical constraints that are visible and relatively stable. Software must often represent complicated rules, exceptions, relationships and interactions. Much of this complexity is essential to the problem rather than accidental. Good engineering manages complexity through abstraction, modularity, clear interfaces and appropriate architecture.

2.5 Software is often custom-built or configured

Reusable components, frameworks, libraries and cloud services improve productivity, but they do not eliminate engineering. Components must be selected, integrated, secured, licensed, tested and maintained. A system assembled from existing services can still fail because of incompatible assumptions or unclear responsibilities.

2.6 Software behaviour depends on its environment

A software system may be correct in isolation but fail when connected to a particular database, network, device, user process or external service. System boundaries and assumptions must therefore be made explicit.

3. Software, Systems and Products

A software product is a deliverable intended for users or customers. It may be a commercial application, an internal business system, an embedded controller or a public digital service. A software system is a set of interacting software and, frequently, hardware, data, networks, people and procedures that together achieve a purpose.

The distinction matters because a system can fail even when individual program modules appear correct. For example, a hospital information system may contain well-tested code but still create unsafe outcomes if user roles, clinical workflows, data quality, training or network availability are ignored.

Common system categories

These categories can overlap. An autonomous vehicle system, for example, is embedded, distributed, real-time, safety-critical and may include artificial intelligence.

4. Stakeholders and Professional Roles

A stakeholder is a person, group or organisation that affects, is affected by, or has an interest in a software system. Stakeholder identification is not a one-time administrative task. Stakeholders may change as the system moves from conception to operation.

Important stakeholder groups include:

• Customers or sponsors: provide funding, strategic direction and business priorities.
• End users: use the system and provide information about real tasks, accessibility and usability.
• Product owners or business analysts: clarify needs, prioritise features and negotiate scope.
• Software engineers and developers: design, implement, integrate and improve software.
• Architects: make decisions about system structure, technology boundaries and quality attributes.
• Test and quality professionals: provide evidence that requirements and quality expectations are satisfied.
• Operations and site-reliability personnel: deploy, monitor, secure and support live services.
• Project managers or delivery leads: coordinate scope, schedule, resources, communication and risks.
• Security, privacy and compliance specialists: identify threats and regulatory obligations.
• Suppliers and third-party service providers: deliver components, platforms or external capabilities.
• Regulators and affected communities: impose legal requirements or experience consequences of system behaviour.

Stakeholders may have conflicting goals. A marketing department may want rapid feature delivery, security staff may require additional controls, users may want simplicity, and finance staff may demand cost reduction. Software engineering involves making such trade-offs explicit and evidence-based.

Professional responsibility

Software professionals should protect the public interest, respect privacy and confidentiality, report significant risks, avoid deceptive claims, maintain competence and give honest estimates. A developer should not silently conceal a known safety defect merely because a release deadline is approaching. Professional judgement includes escalating risks through appropriate channels and documenting decisions.

5. Fundamental Software Engineering Principles

Principles are general rules that guide decisions across languages, tools and development methods.

5.1 Understand the problem before designing the solution

Teams should establish the problem, intended outcomes, constraints, assumptions and affected stakeholders. Building an attractive solution to the wrong problem is a common form of waste.

5.2 Separate concerns

A system is easier to understand when different responsibilities are separated. User-interface code, business rules, persistence and external integration should not be unnecessarily tangled. Separation improves testing and allows one concern to change with less impact on others.

5.3 Use abstraction and modularity

Abstraction focuses attention on essential behaviour while hiding unnecessary detail. Modularity divides a system into components with clear responsibilities and interfaces. A module should aim for high cohesion—closely related responsibilities—and low coupling—limited dependence on other modules.

5.4 Prefer simplicity, but not oversimplification

Simple designs are generally easier to verify and maintain. However, removing necessary controls or domain rules is not simplicity; it is incompleteness. The goal is the simplest design that satisfies legitimate requirements and risks.

5.5 Design for change

Requirements evolve. Stable interfaces, automated tests, configuration management, documentation and loosely coupled components reduce the cost and risk of change.

5.6 Make quality continuous

Quality cannot be added only at the end. Reviews, static analysis, automated tests, security checks, usability evaluation and continuous integration provide earlier evidence. Early detection usually gives a team more options than late detection.

5.7 Manage risk explicitly

A risk is an uncertain event or condition that could affect objectives. Teams identify risks, estimate their likelihood and impact, assign owners, define responses and monitor them. Common responses include avoidance, mitigation, transfer and acceptance.

A simple qualitative risk priority can be represented as:

Here, likelihood and impact are rating scales, such as 1 to 5. This is not a universal physical law; it is a prioritisation aid. Teams must define the scales consistently and should not treat the resulting number as precise probability or financial loss.

5.8 Keep traceability

Traceability links needs to requirements, design elements, implementation, tests and acceptance evidence. It supports impact analysis, audits and demonstration that important obligations have been addressed.

5.9 Automate repeatable work

Builds, tests, deployments, code formatting and security checks are good candidates for automation. Automation reduces manual variation, but automated checks are only valuable when they test meaningful conditions.

6. Software Quality Attributes

A quality attribute is a characteristic that describes how well a system performs or how acceptable it is, rather than only what function it provides. Quality attributes should be stated in observable terms whenever possible.

Quality attributes can conflict. Strong encryption may add processing overhead; extensive logging may consume storage; extreme customisation may reduce maintainability. Architecture and requirements work must therefore identify priorities and trade-offs.

Quality requirements should be testable

“Fast,” “secure” and “easy to use” are useful intentions but weak requirements unless clarified. A stronger statement might be: “For the agreed workload, 95% of catalogue searches shall return results within two seconds.” The statement defines a measurable response-time target, a workload and a percentile. It still requires agreement on measurement conditions, test data and infrastructure.

A quality measure must be interpreted within its context. For example, availability is often expressed as:

Uptime and downtime must use the same time unit and the measurement period must be specified. The formula describes observed or planned service availability; it does not by itself explain causes, user impact or the quality of individual transactions.

7. Software Processes and Lifecycle Thinking

A software process is a set of activities, roles, work products and controls used to develop and operate software. Common activities include:

1. Feasibility and inception: establish the problem, value, constraints and stakeholders.
2. Requirements: discover, analyse, document, prioritise and validate needs.
3. Design and architecture: define structures, interfaces, data and technical decisions.
4. Implementation: construct and integrate components.
5. Verification and validation: check that the product is built correctly and that it solves the intended problem.
6. Deployment: release software into an operational environment.
7. Operation and maintenance: monitor, support, correct, adapt and enhance the system.
8. Retirement: replace or decommission the system while managing data and continuity.

These activities may be arranged using different lifecycle models. A sequential approach can provide strong documentation and approval points where requirements are stable and regulation is important. Iterative and incremental approaches deliver partial capabilities and use feedback to refine the product. Agile methods emphasise frequent delivery, collaboration, adaptation and working software, but they do not remove the need for architecture, documentation, testing or governance.

DevOps extends collaboration between development and operations through automation, monitoring, shared responsibility and continuous delivery practices. The appropriate process depends on risk, domain, regulation, team capability, technical uncertainty and the cost of change.

8. Why Projects Succeed or Fail

Project success is multidimensional. A project may deliver its planned scope but fail to produce user value, or exceed its budget while delivering a strategically important capability. Relevant dimensions include product quality, stakeholder value, schedule, cost, adoption, safety, compliance and maintainability.

Conditions that support success

• a clearly understood problem and measurable objectives;
• active participation by representative users and decision-makers;
• realistic scope, estimates and priorities;
• competent, collaborative and adequately resourced teams;
• early attention to architecture, security, data and quality risks;
• frequent feedback from working software or credible prototypes;
• transparent reporting of progress, uncertainty and defects;
• controlled change and traceability;
• suitable testing and operational preparation; and
• leadership that supports learning rather than hiding bad news.

Common causes of failure

• ambiguous, incomplete or continuously changing requirements;
• unrealistic deadlines or budgets imposed without evidence;
• weak stakeholder engagement;
• excessive complexity and unmanaged technical debt;
• poor communication across technical and business groups;
• inadequate testing, especially of integration and operational scenarios;
• ignoring security, privacy, accessibility or regulatory obligations;
• dependence on unavailable specialists or unreliable suppliers;
• failure to plan deployment, support, data migration and training; and
• measuring activity rather than outcomes.

The so-called “iron triangle” of scope, time and cost is useful for discussing constraints, but quality and value must not be treated as optional leftovers. Reducing testing or security to meet a date may merely move cost and risk into production.

9. Ethics, Security and Governance

Software can amplify decisions and affect people who have little control over its design. Ethical engineering therefore includes:

• collecting only necessary personal data;
• explaining significant automated decisions where appropriate;
• protecting confidentiality, integrity and availability;
• designing for accessibility and non-discrimination;
• testing for foreseeable misuse and harmful failure;
• respecting intellectual-property licences;
• preserving audit records where accountability is required; and
• providing channels for correction, appeal and incident reporting.

Security should be considered throughout the lifecycle. Threat modelling, secure design, least privilege, input validation, dependency management, secrets protection, patching, logging and incident response are engineering activities, not merely administrative controls. Artificial intelligence introduces additional concerns such as biased training data, model drift, privacy leakage, unexplainable outputs and human overreliance. AI tools can assist coding or testing, but generated output requires review, testing, licence checking and security assessment.

Governance defines who may decide, approve, access, change and audit system assets. Good governance is not unnecessary bureaucracy; it creates accountability and helps organisations demonstrate responsible control.

10. Illustrative Case: Student Registration Platform

Background

A college decides to replace paper-based course registration with an online platform. The proposed system will allow students to view available courses, register, pay fees and receive confirmation. It will integrate with student records, finance and identity-management systems.

Problem

An initial project plan focuses almost entirely on building screens and database tables. During a pilot, students discover that prerequisites are interpreted inconsistently, finance records are delayed, and the system becomes unavailable when registration opens. Students with accessibility needs cannot complete the process using a keyboard. Staff also lack clear procedures for correcting failed registrations.

Analysis

The project did not fail because the developers could not write code. It failed because system-level concerns were neglected:

• key stakeholders, including registry staff and accessibility representatives, were not sufficiently involved;
• requirements did not define prerequisite rules, transaction consistency or service capacity;
• external-system assumptions were undocumented;
• availability, usability and accessibility were not treated as explicit quality requirements;
• testing used small artificial workloads and did not represent registration-day behaviour; and
• operational support and exception handling were designed too late.

Approach and solution

The team conducts stakeholder workshops and maps the complete registration process. It defines business rules with examples, identifies system interfaces and establishes measurable quality targets. A thin vertical slice is built: authentication, course selection, prerequisite checking, payment confirmation and notification are tested together.

The team also introduces:

• automated unit, integration and accessibility tests;
• load testing using an expected peak workload and a safety margin;
• transaction and reconciliation procedures for payment failures;
• monitoring of response time, error rate and queue length;
• a staged pilot with student and staff representatives;
• documented recovery and support procedures; and
• a change-control process for registration rules.

Outcome

The revised platform is released in stages. Early pilot feedback identifies a confusing prerequisite message and a payment timeout scenario before full deployment. Registration staff receive an exception-management dashboard, and students receive clearer confirmations. The project takes longer than the original screen-focused estimate, but it reduces operational disruption and improves trust in the service.

Learning points

The case demonstrates that software quality is broader than code correctness. Requirements, architecture, integration, usability, accessibility, security, operations and stakeholder participation must be treated as parts of one system. Early feedback and measurable quality goals reveal risks while they are still manageable.

11. Contemporary Engineering Practice and Best Practices

Modern teams commonly use version control, issue tracking, code review, automated builds, continuous integration, containerisation, cloud platforms, observability tools and infrastructure-as-code. These technologies can improve repeatability and feedback, but tools do not replace sound judgement. A poorly understood process can be automated into failure more quickly.

Recommended practice for diploma-level engineers includes:

1. clarify the problem and identify affected stakeholders;
2. record assumptions, constraints and decisions;
3. express important requirements in testable language;
4. use version control for code, configuration and significant documentation;
5. keep modules focused and interfaces explicit;
6. review code and design decisions with peers;
7. test normal, boundary, failure, security and accessibility cases;
8. automate repeatable verification and deployment tasks;
9. monitor real operation and learn from incidents; and
10. treat maintenance, documentation and technical debt as planned engineering work.

Challenges and Limitations

No method guarantees project success. Requirements may remain uncertain, stakeholders may disagree, estimates may be affected by novelty, and external dependencies may change. Metrics can also mislead: a high number of passed tests does not prove that the right requirements were tested, and a large amount of delivered code does not prove user value. Engineering models and processes must therefore be adapted to context and supported by professional judgement.

Chapter Summary

Software engineering is the disciplined, systematic and measurable development and evolution of software systems. It is broader than programming because it includes problem understanding, requirements, architecture, implementation, testing, deployment, operation and maintenance.

Software is intangible, changeable, complex and dependent on its environment. Systems may be information, embedded, distributed, real-time, safety-critical, business-critical or AI-enabled, and categories can overlap. Stakeholders include users, customers, developers, architects, testers, operators, regulators and affected communities.

Core engineering principles include separation of concerns, abstraction, modularity, simplicity, design for change, continuous quality, explicit risk management and traceability. Quality attributes such as functional suitability, usability, performance, reliability, security, maintainability, accessibility and safety must be prioritised and expressed in testable terms. Project success depends on clear objectives, stakeholder involvement, realistic planning, competent collaboration, early risk management and operational preparation. Failure commonly results from unclear requirements, unrealistic constraints, poor communication, inadequate testing and neglect of quality or governance.

Professional software engineering also requires ethical conduct, privacy protection, secure design, accessibility, accountability and responsible use of emerging technologies. The central lesson is that dependable software is produced by integrating technical skill with disciplined reasoning, communication and responsibility.

Key Terms

Software: Programs, data, configuration and documentation that provide computing capabilities.

Software engineering: A systematic, disciplined and measurable approach to developing and maintaining software.

Software system: Interacting software and often hardware, data, people and procedures serving a purpose.

Stakeholder: A person, group or organisation affected by or interested in a system.

Requirements: Agreed capabilities, constraints and qualities expected of a system.

Architecture: The high-level structure of a system and its major components and relationships.

Abstraction: Representing essential characteristics while hiding unnecessary detail.

Modularity: Dividing a system into manageable components with defined responsibilities.

Cohesion: The degree to which responsibilities within a module belong together.

Coupling: The degree of dependency between modules.

Quality attribute: A characteristic describing how well a system performs or is experienced.

Maintainability: The ease with which software can be analysed, modified and tested.

Reliability: The ability to perform consistently under stated conditions.

Availability: The extent to which a system is accessible when required.

Technical debt: Future cost or risk created by expedient technical decisions.

Traceability: The ability to link related requirements, designs, code, tests and evidence.

Risk: An uncertain event or condition that may affect objectives.

Verification: Checking whether the product was built according to specified requirements.

Validation: Checking whether the product solves the intended user or business problem.

Governance: Structures and rules defining authority, accountability, control and oversight.

Review Questions

Short Answer Questions

1. Define software engineering and explain how it differs from programming.
2. State four characteristics that distinguish software from physical products.
3. What is a stakeholder? Name four software-system stakeholders.
4. Explain the difference between verification and validation.
5. Why should quality requirements be expressed in measurable terms?

Long Answer Questions

1. Explain the major categories of software systems and show how one system can belong to several categories.
2. Discuss separation of concerns, abstraction, modularity, cohesion and coupling as design principles.
3. Describe at least six software quality attributes and explain possible trade-offs between them.
4. Explain the activities of a software lifecycle from inception through retirement.
5. Analyse the major technical, organisational and human causes of software project failure.

Analytical / Application Questions

1. A small clinic wants an appointment system. Identify its key stakeholders, three functional requirements and four quality requirements. Explain how each quality requirement could be evaluated.
2. A team plans to remove security testing to meet a release date. Analyse the risks and recommend a responsible engineering response.
3. Using the student registration case, propose two process changes, two technical changes and two governance changes that would reduce the likelihood of a similar failure.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes software engineering from programming?

A. Software engineering concerns only user-interface design.

B. Programming is unnecessary in software engineering.

C. Software engineering includes requirements, design, testing, operation and maintenance in addition to programming.

D. Programming applies only to physical products.

Correct Answer: C. Software engineering includes requirements, design, testing, operation and maintenance in addition to programming.

Question 2

Which characteristic of software most directly explains why version control and change management are important?

A. Software is intangible.

B. Software is highly changeable.

C. Software is manufactured in factories.

D. Software always operates independently of its environment.

Correct Answer: B. Software is highly changeable.

Question 3

Who is most directly responsible for monitoring and supporting a system in production?

A. Operations or site-reliability personnel

B. End users only

C. The marketing department

D. The project sponsor alone

Correct Answer: A. Operations or site-reliability personnel

Question 4

Which design principle recommends dividing a system into components with clear responsibilities and interfaces?

A. Modularity

B. Randomisation

C. Duplication

D. Obfuscation

Correct Answer: A. Modularity

Question 5

Which quality attribute concerns protection against unauthorised access and data disclosure?

A. Portability

B. Security

C. Usability

D. Scalability

Correct Answer: B. Security

Question 6

What is the primary purpose of traceability?

A. To increase the number of source-code lines

B. To link requirements, design, implementation, tests and evidence

C. To eliminate all stakeholder involvement

D. To replace system testing with documentation

Correct Answer: B. To link requirements, design, implementation, tests and evidence

Question 7

A system is available for 90 hours and unavailable for 10 hours during a measurement period. Using the chapter’s availability formula, what is its availability?

A. 9%

B. 10%

C. 90%

D. 100%

Correct Answer: C. 90%

Question 8

Which condition most strongly supports project success?

A. Deferring all testing until after deployment

B. Changing priorities without recording decisions

C. Active stakeholder participation and measurable objectives

D. Removing operational planning from the project

Correct Answer: C. Active stakeholder participation and measurable objectives

Question 9

What does validation primarily ask?

A. Was the product built according to specified requirements?

B. Does the product solve the intended user or business problem?

C. How many files are in the repository?

D. Which programming language was used?

Correct Answer: B. Does the product solve the intended user or business problem?

Question 10

Why should AI-generated code be reviewed and tested?

A. AI-generated code is automatically compliant with every licence.

B. Generated output can contain defects, security weaknesses or unsuitable assumptions.

C. Testing is unnecessary when code compiles.

D. Review prevents all future software changes.

Correct Answer: B. Generated output can contain defects, security weaknesses or unsuitable assumptions.