Market Research, Data and Highest and Best Use

Introduction

Property valuation is an evidence-based process. The valuer does not merely inspect a building and form an opinion; the valuer interprets a market, assembles evidence, tests alternative uses and reaches a defensible conclusion about value. The quality of that conclusion depends heavily on the quality of market research and data.

Market research identifies the economic environment in which a property competes. Data collection provides evidence about transactions, rents, costs, vacancy, development activity, planning controls and user preferences. Data verification determines whether that evidence is reliable. Data normalization makes unlike observations sufficiently comparable. Highest and best use (HBU) then integrates market, legal, physical and financial analysis to determine the use that produces the greatest value.

This chapter treats HBU as a disciplined decision framework rather than a slogan. A use is not highest and best merely because it produces the largest theoretical income. It must be legally permissible, physically possible, financially feasible and maximally productive.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Define a market and segment it for valuation purposes.
2. Design a structured process for collecting, verifying and storing property data.
3. Normalize transaction, lease, cost and physical data for comparison.
4. Analyse neighbourhood, location, site and building attributes.
5. Apply the four tests of highest and best use.
6. Evaluate residual land value, development feasibility and sensitivity.
7. Recognize ethical, regulatory, technological and methodological limitations in valuation research.

1. The Market as the Object of Analysis

A real estate market is a system of interacting buyers, sellers, occupiers, investors, lenders, developers and regulators. It is not simply a geographic area. A market is defined by the substitution relationships among properties: two properties belong to the same competitive market when a typical purchaser or occupier could reasonably select one instead of the other.

A useful market definition identifies:

• Product: for example, Grade-A offices, student housing, neighbourhood retail or logistics warehouses.
• Geography: metropolitan region, submarket, corridor, district or walkable catchment.
• Participants: owner-occupiers, investors, tenants, developers or public agencies.
• Price or rent segment: prime, secondary, affordable, luxury or distressed.
• Time period: the valuation date and the relevant historical window.
• Decision purpose: lending, acquisition, taxation, financial reporting, litigation or development.

The same property can belong to different markets for different purposes. An office building may compete for tenants in a central business district, for investment capital in a national market and for redevelopment land in a local planning market.

Market segmentation

Segmentation reduces analytical error by comparing genuinely substitutable properties. Common segmentation variables include:

Segmentation should be empirically supported. A valuer should not create a segment merely because it produces a desired result. Evidence may include rent differentials, buyer behaviour, absorption rates, planning policy, brokerage practice and statistical clustering.

Supply, demand and market equilibrium

Demand is affected by population, employment, household formation, income, business formation, credit conditions, user preferences and competing properties. Supply depends on land availability, construction costs, finance, planning controls, developer expectations and existing stock. Market conditions are dynamic: a shortage today may induce new construction that creates surplus later.

Important indicators include vacancy, absorption, inventory, pipeline supply, achieved rents, incentives, days on market, transaction volume and capitalization rates. A high asking rent is not evidence of market rent unless supported by executed leases or carefully adjusted offers.

Absorption over a period may be expressed as:

If 80,000 m² is occupied at the beginning of a year and 92,000 m² at the end, net absorption is 12,000 m². The measure assumes consistent measurement of occupied space and does not by itself reveal whether demand is sustainable.

2. A Data Architecture for Valuation

Types of valuation data

Transaction data include sale price, date, buyer and seller characteristics, financing, conditions of sale, property rights transferred, area, age, quality, occupancy and included personal property.

Lease data include base rent, effective rent, lease term, escalations, rent-free periods, tenant improvements, renewal options, operating expenses, service charges, incentives and break clauses.

Cost data include land cost, hard construction cost, professional fees, finance, marketing, contingencies, taxes and developer profit. Cost sources must state whether figures are nominal or real and whether they include taxes and abnormal works.

Property data include site area, floor area, net lettable area, floor-to-floor height, access, parking, services, condition, environmental characteristics, construction quality and functional utility.

Market and macroeconomic data include interest rates, inflation, employment, household growth, tourism, infrastructure investment, planning changes and investment yields.

The data dictionary

A professional valuation file should contain a data dictionary defining every field. For example, “area” must specify whether it means gross external area, gross internal area, net internal area, net lettable area or saleable area. “Price” must state whether it includes taxes, furniture, business value, development rights or assumed debt.

A useful record also includes source, collection date, valuation date, reliability grade, verification status, adjustment notes and the analyst responsible. Provenance is essential: an unsupported number may be impossible to defend months later.

Data collection hierarchy

The strongest evidence generally comes from:

1. Verified completed transactions and executed leases.
2. Official registries, planning databases and audited financial records.
3. Direct confirmation with parties, agents or property managers.
4. Professional databases with transparent methodology.
5. Market interviews and broker opinions.
6. Advertisements and asking prices.
7. Unverified hearsay or unattributed secondary commentary.

The hierarchy is not absolute. A registry may contain an old nominal price, while a directly confirmed transaction may reveal a material side agreement. The valuer must assess relevance, accuracy, completeness and comparability together.

3. Verification, Cleaning and Normalization

Verification questions

For each comparable, ask:

• Did the transaction actually occur?
• What property rights were transferred?
• Was the sale arm’s length?
• Were there unusual motivations or related parties?
• What was the physical condition on the transaction date?
• Were incentives, tax, furniture or business assets included?
• Is the reported area measured on the same basis?
• Was the property vacant, occupied or subject to a non-market lease?
• What market conditions prevailed at the transaction date?

A transaction should not be rejected solely because it is unusual. It may be useful as evidence of a lower or upper boundary, provided its unusual features are understood and adjusted or separately analysed.

Data cleaning

Cleaning includes removing duplicate records, resolving inconsistent addresses, checking units, correcting dates, identifying impossible values and documenting missing observations. Outliers require investigation rather than automatic deletion. A very high price may reflect superior quality, a development option or an error.

For a variable x, a standardized score is often written as:

where x is the observation, μ is the sample mean and σ is the sample standard deviation. The score is dimensionless and useful for screening, but it does not prove that an observation is erroneous. In small or skewed property samples, the median and interquartile range may be more informative than the mean and standard deviation.

Normalization of transaction evidence

Normalization converts observed evidence into a common analytical basis. Typical adjustments include:

• Time adjustment: reflects market movement between transaction date and valuation date.
• Property-rights adjustment: distinguishes freehold, leasehold, partial interest and subject-to-lease rights.
• Financing adjustment: removes non-market financing effects.
• Conditions-of-sale adjustment: addresses distress, related-party transactions or unusual motivations.
• Location adjustment: reflects accessibility, visibility, surrounding uses and externalities.
• Physical adjustment: reflects size, age, quality, condition, specification and utility.
• Economic-characteristics adjustment: reflects occupancy, lease terms, incentives and operating costs.
• Use or zoning adjustment: recognizes development potential or restrictions.

The adjustment is not a mechanical percentage unless supported by market evidence. A comparable 10% larger than the subject is not necessarily worth 10% less per square metre; size effects may be nonlinear.

For a comparable sale price P adjusted sequentially by factors a₁, a₂, …, aₙ, the adjusted price is:

Each aᵢ is expressed as a decimal, such as −0.05 for a 5% downward adjustment. This multiplicative approach assumes each adjustment acts on the already-adjusted amount and that the factors are sufficiently separable. If adjustments are estimated as absolute monetary amounts, they should instead be added in a consistent unit basis.

Worked example: comparable normalization

A verified office sale transferred for $12,000,000. The comparable is 4% inferior in location, 3% superior in physical condition and requires a 6% upward market-time adjustment. Assuming sequential percentage adjustments:

The location adjustment is positive because the comparable is inferior to the subject. The condition adjustment is negative because the comparable is superior. The result is not a fact; it is an analytical estimate whose credibility depends on the evidence supporting each factor.

Lease normalization

Quoted rent may be misleading. Effective rent incorporates incentives such as rent-free periods and tenant-improvement contributions. If a tenant pays monthly base rent R for n months, receives f rent-free months and pays an incentive I, a simplified effective monthly rent is:

This formula assumes constant nominal rent, no discounting and an incentive paid at the same economic time as rent. For long leases, a present-value approach is preferable because timing matters. Lease comparability also requires matching expense recoveries, escalation, renewal rights, break options and fit-out obligations.

4. Neighbourhood, Location and Site Analysis

Location analysis moves from broad context to property-specific attributes.

Regional and neighbourhood analysis

Regional analysis considers employment, infrastructure, demographic change, policy, migration, economic diversification and competing centres. Neighbourhood analysis examines land-use mix, population density, household profile, retail services, schools, public realm, safety, environmental quality, development pipeline and social change.

A neighbourhood is not simply “good” or “bad.” Its suitability depends on the property use. A high-density transit location may be excellent for apartments but unsuitable for low-density logistics. Externalities can be positive—transit, parks, employment—or negative—noise, congestion, pollution, flooding or incompatible uses.

Site and building analysis

Site analysis should address:

• ownership and title constraints;
• dimensions, shape, frontage and topography;
• access, visibility and circulation;
• utilities and servicing capacity;
• geotechnical, environmental and flood conditions;
• setbacks, height, density, coverage and parking controls;
• easements, covenants and rights of way;
• heritage, conservation and protected trees;
• building condition, adaptability and obsolescence.

GIS, satellite imagery, cadastral records and planning portals can improve spatial analysis, but they do not replace legal review or physical inspection. A map may show a road connection that is not legally accessible.

5. Highest and Best Use

Highest and best use is the reasonably probable use of property that is legally permissible, physically possible, financially feasible and maximally productive. The analysis is performed both as vacant and as improved where relevant.

The four tests are sequential but interactive:

1. Legally permissible: Is the use allowed by zoning, title, lease, environmental regulation, building regulation and other controls? A use requiring a speculative rezoning should not normally be treated as currently permissible, although its probability and timing may be analysed separately.
2. Physically possible: Can the site support the use considering size, shape, access, topography, services, hazards, construction technology and environmental constraints?
3. Financially feasible: Would the use generate a return sufficient to justify land, construction, finance, risk and developer profit? Feasibility is not established by positive gross revenue alone.
4. Maximally productive: Among feasible alternatives, which produces the highest value of the land or property, usually after recognizing risk and timing?

The tests must be applied to realistic alternatives, not every imaginable use. A 50-storey tower may be physically possible in an engineering sense but financially infeasible, legally prohibited or inconsistent with market demand.

Residual land value

For a development option, a simplified residual land value is:

Gross development value is the market value of completed units or stabilized income-producing property. Total development costs may include construction, professional fees, planning, finance, marketing, taxes, infrastructure, contingencies and abnormal costs. The developer’s required profit compensates for risk and entrepreneurial effort.

This residual is highly sensitive to assumptions. It is not automatically the price a developer should pay: acquisition costs, timing, tax and risk adjustments must be treated consistently.

Income capitalization and discounted cash flow

For a stabilized property with sustainable net operating income (NOI), a simple capitalization indication is:

where V is value, NOI is annual stabilized net operating income and R is the capitalization rate expressed as a decimal. The formula assumes a stabilized, perpetual income stream, consistent risk and a capitalization rate that appropriately reflects growth and market expectations. It is not suitable without modification for irregular cash flows, major lease rollover or development property.

For a discounted cash flow (DCF) analysis:

where V₀ is value at the valuation date, CFₜ is net cash flow in period t, k is the discount rate, n is the explicit forecast period and TVₙ is terminal value at the end of period n. All cash flows must be defined consistently as nominal or real and before or after tax. The discount rate must match the risk and cash-flow definition.

Maximum productivity and land residual logic

A use may have the highest completed-property value but not the highest land value. For example, luxury apartments may generate high sales revenue but require unusually high construction costs, marketing periods and risk. A smaller, lower-risk scheme may produce a greater residual land value.

6. A Structured HBU Workflow

Figure: Iterative workflow for highest and best use analysis.

A defensible workflow is:

1. Define the valuation purpose, date and property rights.
2. Delineate the market and collect primary and secondary evidence.
3. Inspect the property and verify legal and physical constraints.
4. Identify realistic alternative uses.
5. Test legal permissibility and physical possibility.
6. Prepare preliminary development or operating scenarios.
7. Estimate revenues, costs, timing, risk and required return.
8. Test financial feasibility and residual value.
9. Compare feasible alternatives for maximum productivity.
10. Document assumptions, rejected alternatives, sensitivity and conclusion.

7. Worked HBU Application

Consider an illustrative, fictional 4,000 m² urban site. Three alternatives are considered: retain the existing low-rise office, develop apartments, or develop a small mixed-use scheme. Planning review confirms that all three are legally permissible. Site investigation confirms that each is physically possible.

The analysis produces the following estimates:

The mixed-use alternative has the highest residual land value and is therefore initially maximally productive. However, the conclusion requires further scrutiny. Its costs may be more volatile, construction may take longer and retail absorption may be uncertain. A sensitivity analysis should vary sales prices, construction costs, interest rates, programme duration and absorption. If a modest adverse change reduces the mixed-use residual below the apartment residual, the valuer should explain the risk rather than present $9.0 million as a precise fact.

The existing office may still be the appropriate HBU as improved if demolition is restricted, the current lease is unusually valuable, or redevelopment risk is not adequately compensated. HBU is a market conclusion, not simply an engineering or architectural conclusion.

8. Sensitivity, Scenario and Risk Analysis

Property forecasts are uncertain because prices, rents, costs, interest rates and regulatory decisions change. Sensitivity analysis changes one assumption at a time to identify important value drivers. Scenario analysis changes several related assumptions to describe coherent states such as base, downside and upside markets.

A simple percentage sensitivity of value to an input x can be expressed as:

where ∆V is the change in indicated value and ∆x is the change in the selected input. This is a local measure, not a universal causal relationship. For a DCF, changing the discount rate and terminal assumptions together may be more realistic than changing either alone.

Monte Carlo simulation may be used when probability distributions for key variables are credible. It should not create false precision: simulated outputs are only as defensible as the distributions, correlations and model structure supplied by the analyst.

9. Technology, AI and Data Governance

Digital tools support, but do not replace, professional judgment. GIS can measure accessibility and land-use context; automated valuation models can detect patterns; remote sensing can identify construction and land-use change; natural-language tools can classify leases and extract clauses; and databases can monitor transactions and rents.

Important controls include:

• documenting data sources and model versions;
• checking geocoding, area measurement and duplicate records;
• separating observed facts from model estimates;
• testing for sample-selection and geographic bias;
• validating predictions against out-of-sample evidence;
• protecting confidential tenant and transaction information;
• preserving an audit trail of human review and overrides.

An AI-generated comparable or extracted lease term must be independently verified. A model trained on asking prices may systematically overstate market value. A model trained on historic transactions may understate value in a rapidly changing market. Explainability and governance are particularly important in lending, taxation and legal disputes.

10. Ethical, Regulatory and Professional Issues

Valuers must act with competence, independence, confidentiality and integrity, subject to the applicable professional standards and jurisdictional law. They should disclose conflicts of interest, material assumptions, limitations on inspection and reliance on third-party information.

Common ethical risks include selective comparable choice, unsupported adjustments, double counting of benefits, concealment of adverse information, pressure to meet a target value and inappropriate certainty in reporting. The remedy is not merely a disclaimer. It is transparent evidence, reasoned analysis and a clear distinction between fact, assumption and opinion.

Regulatory requirements may concern zoning, environmental approval, building safety, accessibility, heritage, taxation, anti-money-laundering controls, data protection and professional licensing. Legal permissibility should be confirmed by appropriately qualified legal or planning professionals where the issue is material.

11. Limitations and Best Practice

Market data are often incomplete, stale, heterogeneous and subject to survivorship bias. Transaction samples may be small; leases may contain confidential incentives; cost data may omit abnormal conditions; and asking prices may be strategically set. The valuer should avoid presenting a narrow point estimate where a range is more honest.

Best practice is to:

• define the market before selecting comparables;
• use multiple independent sources;
• verify material facts directly where possible;
• match measurement bases and property rights;
• explain every material adjustment;
• distinguish nominal from real values and gross from net areas;
• use sensitivity and scenario analysis;
• test both as-vacant and as-improved use where relevant;
• record rejected evidence and rejected HBU alternatives;
• communicate uncertainty in plain professional language.

Chapter Summary

Market research identifies the competitive environment in which a property operates. Effective analysis requires explicit segmentation by product, geography, participants, quality, tenure, risk and time. Valuation data include transactions, leases, costs, physical attributes, market indicators and legal information; each observation requires provenance, verification and a defined measurement basis.

Normalization makes evidence comparable by addressing time, rights, financing, conditions of sale, location, physical characteristics and economic terms. Lease incentives and area definitions are particularly important sources of error. Neighbourhood and site analysis should connect regional trends with property-specific legal, physical and environmental conditions.

Highest and best use is tested through legal permissibility, physical possibility, financial feasibility and maximum productivity. Residual land value, capitalization and DCF methods can support the analysis, but their assumptions must be consistent and sensitivity-tested. Technology improves collection and analysis, while professional judgment, independence, auditability and ethical governance remain essential.

Key Terms

Market: A system of buyers, sellers, occupiers and other participants interacting for a defined property product and location.

Market segmentation: Division of a broad market into groups with meaningful competitive and economic similarities.

Comparable: A property or transaction used as evidence because it shares relevant characteristics with the subject.

Provenance: The documented origin, collection date and verification history of data.

Normalization: Conversion of unlike observations to a common analytical basis.

Arm’s-length transaction: A transaction between independent, informed parties without abnormal compulsion.

Effective rent: Rent after considering incentives, rent-free periods and other economic concessions.

Net absorption: Change in occupied stock over a specified period.

Externality: A benefit or cost imposed by surrounding conditions on a property.

Physical possibility: The capacity of a site and its improvements to support a proposed use.

Legal permissibility: Compliance of a proposed use with applicable law, planning and title controls.

Financial feasibility: The ability of a proposed use to generate an adequate risk-adjusted return.

Maximum productivity: The feasible use producing the greatest value contribution.

Highest and best use: The reasonably probable legal, physical, financially feasible and maximally productive use.

Residual land value: Value remaining for land after development value, costs and required profit.

Capitalization rate: A rate used to convert stabilized income into an indication of value.

Discounted cash flow: Valuation by discounting forecast cash flows and terminal value to present value.

Sensitivity analysis: Examination of value changes resulting from changes in selected assumptions.

Data governance: Policies and controls ensuring data quality, security, accountability and appropriate use.

Obsolescence: Loss of utility or value caused by functional, economic, technological or external factors.

Review Questions

Short Answer Questions

1. What factors should be specified when defining a real estate market?
2. Why is data provenance important in valuation?
3. Distinguish gross rent from effective rent.
4. State the four tests of highest and best use.
5. What is the purpose of sensitivity analysis?

Long Answer Questions

1. Explain how transaction data should be verified and normalized before use in the sales comparison approach.
2. Discuss the relationship between neighbourhood analysis and property-specific site analysis.
3. Explain residual land value and identify the principal cost and risk components that should be included.
4. Critically evaluate the use of capitalization and discounted cash flow methods in HBU analysis.
5. Discuss ethical and governance risks associated with automated valuation models and AI-assisted research.

Analytical / Application Questions

1. A comparable sale is physically superior to the subject but was completed eight months earlier in a rising market. Design an adjustment framework and identify the evidence needed to support it.
2. A site permits apartments and offices. Apartments have higher gross revenue, but offices have lower construction risk and faster absorption. Explain how you would determine the maximally productive use.
3. A proposed mixed-use scheme has a residual land value only 3% above the apartment alternative, while its costs and programme are substantially more uncertain. Recommend an analytical and reporting approach for this decision.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a properly defined real estate market?

A. All properties within the same municipal boundary

B. Properties that are reasonably substitutable for a defined group of participants

C. Only properties that sold during the previous month

D. All properties having the same construction materials

Correct Answer: B. Properties that are reasonably substitutable for a defined group of participants

Question 2

Which item is most directly part of data provenance?

A. The valuer’s preferred capitalization rate

B. The decorative style of the building

C. The source and verification history of an observation

D. The owner’s future renovation preference

Correct Answer: C. The source and verification history of an observation

Question 3

A comparable sold for $10,000,000. A 5% upward time adjustment and a 4% downward location adjustment are applied sequentially. What is the adjusted price?

A. $9,500,000

B. $10,080,000

C. $10,100,000

D. $10,920,000

Correct Answer: B. $10,080,000

Question 4

Which adjustment addresses rent-free periods and tenant-improvement concessions?

A. Effective-rent normalization

B. Topographic adjustment

C. Title adjustment

D. Geocoding adjustment

Correct Answer: A. Effective-rent normalization

Question 5

Which is not one of the four conventional HBU tests?

A. Legally permissible

B. Physically possible

C. Financially feasible

D. Politically popular

Correct Answer: D. Politically popular

Question 6

A development has a gross development value of $30 million, costs excluding land of $21 million and required developer profit of $3 million. What is the simplified residual land value?

A. $3 million

B. $6 million

C. $9 million

D. $54 million

Correct Answer: B. $6 million

Question 7

The capitalization formula V = NOI ÷ R is most appropriate when the property has:

A. Highly irregular development cash flows

B. A stabilized and sustainable income stream

C. No measurable operating income

D. Only speculative rezoning potential

Correct Answer: B. A stabilized and sustainable income stream

Question 8

If an alternative use is legally permissible and physically possible but cannot generate a sufficient risk-adjusted return, it fails which HBU test?

A. Market segmentation

B. Financial feasibility

C. Data provenance

D. Measurement consistency

Correct Answer: B. Financial feasibility

Question 9

Why should an unusual transaction not automatically be deleted from a comparable dataset?

A. All unusual transactions are necessarily superior evidence

B. It may reveal a market boundary if its unusual features are understood

C. Outliers never affect valuation conclusions

D. Deletion is prohibited in all valuation standards

Correct Answer: B. It may reveal a market boundary if its unusual features are understood

Question 10

Which is an appropriate governance control for an AI-assisted valuation model?

A. Accepting model output without human review

B. Using only asking prices because they are easy to collect

C. Maintaining source records, model versions and validation evidence

D. Removing all low-value observations automatically

Correct Answer: C. Maintaining source records, model versions and validation evidence