Search Engine Optimization

Introduction

Search engine optimization (SEO) is the planned improvement of a website’s visibility, usefulness and credibility in unpaid search results. It is a central digital marketing activity because search engines connect people expressing a need with organisations offering information, products or services. Effective SEO is not simply the insertion of popular words into web pages. It combines audience research, content design, website engineering, reputation management, measurement and ethical communication.

Search results are produced by complex systems that discover, interpret, evaluate and rank content. Their precise ranking formulas are proprietary and continually changing. Consequently, SEO professionals should focus on durable principles: satisfying search intent, publishing accurate and original information, making websites accessible to users and crawlers, and earning legitimate trust.

This chapter explains how search engines work and provides a practical framework for keyword research, on-page, technical, local and off-page SEO. It also considers quality, authority, ethical practice and the growing influence of artificial intelligence (AI) on search.

Learning objectives

By the end of this chapter, you should be able to:

1. Explain crawling, indexing, retrieval and ranking.
2. Conduct basic keyword research and classify search intent.
3. Apply on-page SEO principles to a web page.
4. Identify important technical, local and off-page SEO activities.
5. Distinguish quality, relevance and authority from manipulative optimization.
6. Develop a simple SEO plan and evaluate its performance.
7. Recognise how AI-generated content and AI-mediated search affect SEO practice.

1. How search engines work

A search engine attempts to return results that are useful, relevant, reliable and appropriate to a user’s query. Four related processes are important.

Crawling

Crawling is the automated discovery of web resources by programs commonly called crawlers, spiders or bots. A crawler follows links, reads sitemaps and revisits known URLs. It may also discover URLs through references from other websites. A site’s robots.txt file can communicate crawling preferences, although it is not a security mechanism and does not guarantee that a URL will remain absent from search results.

Crawling is affected by accessibility, server availability, internal linking, duplicate URLs and the site’s perceived importance. A page that cannot be reached through links or a sitemap may be discovered slowly or not at all.

Indexing

Indexing is the process of analysing and storing information about a page. The search engine may interpret text, headings, images, structured data, language, canonical relationships and page experience. Indexing does not guarantee ranking. A page can be technically accessible but excluded, considered duplicative, or judged insufficiently useful for a particular query.

A noindex directive is different from robots.txt: it tells a compliant search engine not to include a page in its index, whereas robots.txt primarily controls crawling access. Sensitive information should be protected through authentication, not either mechanism.

Retrieval and ranking

When a user submits a query, the engine retrieves potentially relevant documents from its index and orders them using many signals. These may include textual relevance, content quality, links, location, device, freshness, usability and the meaning of the query. Results can also include maps, images, videos, news, featured extracts and other search features.

Ranking is query-dependent. A national newspaper may be authoritative for a current event but irrelevant to a local plumber’s service query. SEO therefore aims not at a universal position but at visibility for appropriate audiences and valuable searches.

Figure: The principal stages through which web content becomes a search result.

2. Keywords and search intent

A keyword is a word or phrase used to describe a search need. In practice, marketers investigate queries rather than treating keywords as isolated strings. Keyword research identifies the language, problems and priorities of a target audience.

Useful sources include search-engine suggestions, internal site-search data, customer interviews, sales questions, competitor content, keyword tools and performance data from search consoles. Tool estimates are directional rather than exact: search volumes vary by country, device, season and data source.

Keywords can be classified by length and purpose. Broad terms such as “shoes” may have high demand but unclear intent. More specific terms such as “waterproof walking shoes for women” usually communicate a clearer need and may be easier to serve. Brand terms refer to an organisation; non-brand terms do not. Informational, commercial-investigation, transactional and navigational categories are also useful.

Search intent

Search intent is the likely purpose behind a query. Four practical categories are:

Intent is not always explicit. “Best accounting software” may indicate comparison, whereas “buy accounting software” is more transactional. Search-result inspection is valuable: if the results are mainly videos, local listings or product pages, that provides evidence about the format and intent the engine currently associates with the query.

A basic keyword map assigns a primary topic and intent to each important URL. It prevents several pages competing for the same purpose, a problem often called keyword cannibalisation. Secondary terms should be included naturally where they help explain the topic, not inserted mechanically.

3. On-page SEO

On-page SEO improves the content and presentation of an individual page.

Content quality and topical relevance

A strong page addresses a defined audience and purpose. It should provide accurate, sufficiently complete and original information, demonstrate appropriate expertise, and make important facts easy to verify. Content should be written for people first. Repeating a phrase unnaturally, producing thin pages at scale or copying competitors does not create genuine relevance.

A useful content brief identifies the audience, primary question, intent, desired action, supporting questions, evidence, internal links and content format. Headings should represent the logical structure of the document. Paragraphs, lists, tables and examples improve comprehension and accessibility.

Titles, descriptions and headings

The title element helps users and search engines understand the page topic and often influences the clickable result heading. It should be distinctive, accurate and concise. A meta description is not a guaranteed ranking factor, but a clear description can influence whether a searcher chooses the result. The main visible heading should describe the page, and subheadings should organise the content hierarchically.

URLs, images and links

Readable URLs communicate topic and site structure. Descriptive image filenames and alternative text support accessibility and image understanding. Alternative text should describe the meaningful content or function of an image; decorative images can generally use empty alternative text. Internal links help users navigate and distribute contextual understanding. Anchor text should describe the destination rather than rely on vague wording such as “click here.”

Structured data is machine-readable markup that may help a search engine interpret entities such as products, events, organisations or reviews. It does not guarantee enhanced search results, and it must accurately represent visible page content.

4. Technical SEO

Technical SEO concerns the ability of users and search systems to access, interpret and use a site.

Important checks include:

• Crawlability: valid links, sensible navigation, functioning server responses and an accurate XML sitemap.
• Indexability: appropriate status codes, canonical signals, prevention of accidental noindex, and control of duplicate or parameter-based URLs.
• Mobile usability: responsive layouts, readable text, usable controls and content parity across devices.
• Performance: efficient images, reduced unnecessary scripts, suitable hosting and stable layout. Performance supports user experience, but it should not replace useful content.
• Security: HTTPS, secure administration, timely updates and protection of personal data.
• Accessibility: semantic HTML, keyboard access, colour contrast, captions and meaningful alternative text.
• Information architecture: clear categories and a shallow, logical path to important pages.

HTTP status codes communicate server outcomes. A successful page commonly returns 200; a permanent move uses 301; a missing resource returns 404. Redirect chains, broken links and soft 404 pages can confuse users and crawlers. Canonical tags help indicate a preferred version among substantially similar URLs, but they are signals rather than absolute commands.

Technical audits should prioritise business impact. Fixing a blocked checkout or thousands of accidental duplicate URLs is generally more urgent than changing a minor metadata detail.

5. Local SEO

Local SEO helps an organisation appear for geographically relevant searches, including searches that imply a location without naming one. It is especially important for retailers, clinics, restaurants, tradespeople and service businesses.

Core activities include maintaining an accurate business profile on relevant search and map services, using a consistent name, address and telephone number, specifying categories and service areas, publishing useful local content, and earning legitimate local references. Reviews can influence consumer confidence and local visibility. Businesses should request honest feedback, respond professionally and never buy, fabricate or selectively manipulate reviews.

A location page should provide genuinely useful information: address, hours, access details, services, local relevance and contact options. Creating near-identical pages for many towns without serving those places is low-quality doorway-page practice.

6. Off-page SEO, authority and trust

Off-page SEO concerns signals and relationships outside a website. The most valuable links are editorially earned links from relevant, credible sources. They may result from original research, useful tools, expert commentary, partnerships, community involvement or newsworthy work.

Not all links have equal value, and a large quantity does not automatically indicate authority. Relevance, source credibility, context, editorial judgment and diversity of referring domains matter. Digital public relations, relationship building and high-quality content are more sustainable than automated link acquisition.

Manipulative practices include buying links intended to pass ranking credit, participating in link schemes, excessive reciprocal links, hidden links and large-scale low-value guest posting. Paid or sponsored links should be clearly disclosed and use appropriate link attributes where required. A site should not ask unrelated websites to publish artificial endorsements merely to improve rankings.

Authority is multidimensional. It may reflect demonstrated knowledge, reputation, citations, organisational legitimacy, expert review, transparent authorship and user trust. A new organisation can build authority through consistent evidence and service quality; it cannot manufacture durable authority through metrics alone.

7. Measurement and decision-making

SEO measurement should connect visibility with user and organisational outcomes. Useful indicators include indexed pages, impressions, clicks, click-through rate (CTR), average position, organic sessions, engaged visits, leads, revenue, conversion rate and assisted conversions. Each metric has limitations. Position averages can hide differences by query and location, while traffic without relevant engagement may have little value.

CTR is commonly calculated as follows:

Here, C is the number of clicks and I is the number of impressions. The formula assumes both measures refer to the same period, search property, audience and query set. For example, 240 clicks from 12,000 impressions gives (240 ÷ 12000) × 100% = 2%. CTR describes search-result response; it does not by itself measure sales quality.

Conversion rate can be defined as:

The denominator must be stated because organisations may use sessions, users, clicks or leads. A sound report segments by landing page, intent, device, location and brand/non-brand status, and compares an appropriate time period. Correlation is not proof of causation: a ranking increase may coincide with seasonality, advertising, publicity or a website redesign.

A practical SEO cycle is: audit → prioritise → implement → measure → learn. Prioritisation can consider expected benefit, confidence, effort, risk and strategic fit. The framework is a decision aid, not a scientifically validated ranking formula.

8. AI-influenced search and content

AI affects SEO in two related ways. First, marketers use generative systems to assist with research, outlines, classification, translation or drafting. Second, search interfaces increasingly use machine-generated summaries, conversational answers and entity-based retrieval. Users may receive an answer without visiting a traditional results page, changing the meaning of visibility.

AI assistance does not remove the need for human accountability. Generated content can contain factual errors, invented sources, outdated claims, biased language, plagiarism or confidential information. Organisations should establish review processes, disclose material use where appropriate, protect personal and proprietary data, and retain an accountable editor. AI-generated text that adds no original value and is produced mainly to manipulate rankings is poor practice regardless of its fluency.

To remain useful in AI-mediated search, content should state facts clearly, demonstrate first-hand experience where relevant, cite reliable sources, maintain identifiable authorship, use structured information accurately and answer real user questions. Distinctive research, tools, data, examples and expert judgment are difficult to replace with generic text. SEO professionals should also monitor how their brand and content are represented in answer interfaces, while recognising that such interfaces may change rapidly and may not provide complete measurement data.

9. Ethics, governance and limitations

SEO operates within legal, professional and social responsibilities. Marketers should avoid deception, hidden text, cloaking, malware, impersonation, fabricated reviews and discriminatory targeting. Claims must be truthful and appropriately evidenced. Privacy laws and platform rules may apply when collecting search, location or behavioural data. Accessibility is both an inclusion obligation and a quality practice, not merely a ranking tactic.

Search engines are imperfect. Results can reflect commercial incentives, algorithmic bias, spam, language inequality and unequal access to resources. Rankings are not objective measures of truth. A responsible marketer therefore treats search visibility as one channel within a broader communication strategy and refers users to qualified professionals when content concerns health, finance, law or safety.

Applied case: GreenRoute Cycle Repairs

Background

GreenRoute is an illustrative fictional mobile bicycle-repair service operating in two urban districts. Its website receives few organic enquiries, although customers frequently ask questions such as “repair a puncture near me” and “how much does a bicycle service cost?”

Problem analysis

An audit finds that the home page targets every service, local information is incomplete, service pages have similar titles, images lack alternative text, and the booking form is difficult to use on mobile devices. The business has several genuine customer reviews but has not maintained its map profile. Competitors rank with pages that clearly explain prices, service areas and response times.

The team maps intents: informational pages answer maintenance questions; commercial pages explain service options and prices; transactional pages support booking; local pages describe the districts actually served. It checks search results and interviews customers rather than selecting terms solely by estimated volume.

Approach

GreenRoute creates separate, useful pages for puncture repair, annual servicing and emergency call-outs. Each page has a clear title, descriptive heading, concise service explanation, transparent limitations, internal links and a booking call to action. It improves mobile form usability, compresses images, adds accurate business information and requests honest reviews after completed jobs. A short guide explains how to identify a puncture, with safety advice and a clear recommendation to seek professional help when necessary.

The business earns local links by contributing maintenance information to a cycling association and sponsoring a community safety event. It does not buy reviews or create pages for districts it does not serve.

Outcome and learning points

After a measurement period, the team evaluates impressions, relevant clicks, bookings, completed jobs and customer quality by landing page. The central lesson is that SEO improvement came from aligning intent, content, technical access, local credibility and service experience. No single keyword or ranking position adequately represents success.

Best-practice checklist

1. Define the audience, business purpose and search intents.
2. Map each important intent to the most appropriate URL.
3. Write original, accurate and genuinely useful content.
4. Make pages crawlable, indexable, secure, mobile-friendly and accessible.
5. Use titles, headings, links, images and structured data accurately.
6. Maintain trustworthy local information and request honest reviews.
7. Earn relevant recognition rather than manipulating links.
8. Measure outcomes, segment data and avoid vanity metrics.
9. Review AI-assisted work for accuracy, originality, privacy and bias.
10. Improve continuously through evidence rather than chasing every algorithm rumour.

Chapter Summary

Search engines discover pages through crawling, analyse and store them through indexing, and retrieve and rank them in response to queries. SEO is therefore a coordinated practice rather than a keyword-placement exercise. Keyword research identifies audience language, while search-intent analysis determines the content and experience most likely to satisfy a need. On-page SEO improves relevance and clarity; technical SEO improves access, interpretation, performance and usability. Local SEO supports geographically relevant discovery, while off-page SEO develops reputation through legitimate relationships and editorial recognition.

Quality, expertise, transparency and user value provide more durable foundations than manipulation. Measurement should connect impressions and clicks with meaningful outcomes such as enquiries, bookings or sales. AI is changing both content production and search interfaces, but it increases rather than removes the need for human review, evidence, accountability and ethical governance.

Key Terms

Crawling: Automated discovery of web resources by search-engine programs.

Indexing: Analysing and storing information about discovered pages.

Ranking: Ordering retrieved results for a particular query.

Search intent: The likely purpose or need behind a search query.

Keyword: A word or phrase representing a search topic or need.

Keyword map: A plan connecting search topics and intents to specific URLs.

On-page SEO: Optimization of content and elements on an individual page.

Technical SEO: Optimization of access, structure, performance and interpretation.

Crawlability: The extent to which a crawler can discover and access content.

Indexability: The extent to which content can be included in a search index.

Canonical URL: The preferred version of substantially similar URLs.

Structured data: Machine-readable information describing page entities or attributes.

Local SEO: Optimization for geographically relevant searches.

Off-page SEO: Activities and signals occurring outside the organization’s website.

Authority: Perceived credibility, expertise and legitimacy of a source.

Search feature: A result format such as a map, image, video or enhanced extract.

Click-through rate: Clicks divided by impressions, expressed as a percentage.

Conversion: A valuable completed action, such as a booking, enquiry or purchase.

Keyword cannibalisation: Multiple pages competing for the same search purpose.

Generative AI: Artificial intelligence that produces text, images, code or other content.

Review Questions

Short Answer Questions

1. What is the difference between crawling and indexing?
2. Define search intent and name four common intent categories.
3. State two purposes of internal links.
4. Why is robots.txt not a suitable method for protecting confidential information?
5. What does click-through rate measure?

Long Answer Questions

1. Explain how a search engine moves from web-page discovery to ranked results.
2. Discuss how keyword research and search-intent analysis should shape a content plan.
3. Compare on-page SEO and technical SEO, giving practical examples of each.
4. Explain how a local business can build visibility without using deceptive practices.
5. Discuss the opportunities and risks of generative AI in SEO.

Analytical / Application Questions

1. A service website has high impressions but few relevant enquiries. Diagnose three possible causes and propose an evidence-based improvement plan.
2. Design a keyword map for a local dental clinic containing one informational, one commercial-investigation and one transactional topic. Justify the page format and intent for each.
3. An agency proposes buying hundreds of links and creating AI-generated pages for every suburb in a city. Evaluate the proposal using quality, technical, ethical and measurement criteria, then recommend an alternative strategy.

Multiple Choice Questions with Answers

Question 1

Which process involves discovering web pages by following links and reading sitemaps?

A. Ranking

B. Crawling

C. Conversion

D. Indexing

Correct Answer: B. Crawling

Question 2

A query such as “how to repair a leaking tap” is primarily which type of intent?

A. Informational

B. Navigational

C. Transactional

D. Branded

Correct Answer: A. Informational

Question 3

What is the main purpose of a keyword map?

A. To calculate advertising expenditure

B. To assign search topics and intents to appropriate URLs

C. To hide pages from crawlers

D. To generate customer reviews

Correct Answer: B. To assign search topics and intents to appropriate URLs

Question 4

Which practice best represents ethical off-page SEO?

A. Buying hundreds of unrelated links

B. Hiding links in page backgrounds

C. Earning a relevant editorial link through useful research

D. Creating fake reviews on local platforms

Correct Answer: C. Earning a relevant editorial link through useful research

Question 5

A page returns a 301 status code. What does this normally indicate?

A. The page is permanently redirected

B. The page is temporarily unavailable

C. The page has no images

D. The page is blocked by a password

Correct Answer: A. The page is permanently redirected

Question 6

A page receives 500 clicks from 20,000 impressions. What is its CTR?

A. 0.25%

B. 2.5%

C. 4%

D. 25%

Correct Answer: B. 2.5%

Question 7

Which statement about structured data is most accurate?

A. It guarantees the page will rank first

B. It replaces the need for useful content

C. It helps machines interpret content but does not guarantee enhanced results

D. It prevents all duplicate URLs

Correct Answer: C. It helps machines interpret content but does not guarantee enhanced results

Question 8

Which action is most appropriate for a local business seeking genuine reviews?

A. Purchase five-star reviews

B. Ask every customer to submit only positive wording

C. Request honest feedback after completed service

D. Create reviews using staff accounts

Correct Answer: C. Request honest feedback after completed service

Question 9

Why should AI-assisted SEO content receive human review?

A. AI output is always too short

B. AI may produce errors, bias, invented sources or privacy risks

C. Search engines cannot index edited content

D. Human review automatically guarantees a top ranking

Correct Answer: B. AI may produce errors, bias, invented sources or privacy risks

Question 10

Which metric most directly represents a completed valuable action such as a booking?

A. Conversion

B. Impression

C. Average position

D. Crawl request

Correct Answer: A. Conversion