Business Intelligence, Data Visualisation and Analytics

Introduction and Context

Banks, insurers and other financial institutions generate data continuously: account transactions, loan applications, claims, customer interactions, market prices, digital sessions, complaints and regulatory reports. Data becomes valuable only when it is converted into reliable information and used to support decisions. Business intelligence (BI) is the organised use of data, technology and analytical methods to understand business performance and support managerial action. Data visualisation presents information through charts, tables, maps and interactive dashboards. Business analytics applies statistical, computational and business reasoning to explain events, forecast outcomes and recommend actions.

For a banking, finance and insurance professional, these capabilities are not merely technical. They affect credit approval, fraud detection, liquidity management, customer retention, claims settlement, branch performance and regulatory compliance. However, attractive charts do not guarantee sound decisions. The analyst must establish data quality, select appropriate measures, interpret uncertainty and protect confidentiality.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Explain the relationship among business intelligence, data management, visualisation and analytics.
2. Prepare, clean and integrate data from operational and external sources.
3. Distinguish descriptive, diagnostic, predictive and prescriptive analytics.
4. Design meaningful KPIs, scorecards and dashboards for banking, finance and insurance.
5. Select suitable visualisations and interpret them critically.
6. Apply analytical results to operational, customer, risk and financial decisions.
7. Identify data-quality, ethical, privacy, security and governance risks.

1. From Data to Decision

A useful information chain is:

Data → Information → Insight → Decision → Action → Outcome

Raw data consists of recorded observations, such as a transaction amount of $250. Information adds context, such as the customer, date, channel and currency. Insight identifies meaning—for example, that card-not-present transactions increased sharply in a particular region. A decision determines what to do, such as introducing an additional authentication step. Action implements the decision, and the outcome must be monitored.

BI commonly has four layers:

A data warehouse is a structured repository designed mainly for reporting and analysis. A data lake stores large volumes of structured and unstructured data in relatively raw form. A data mart is a subject-specific analytical store, such as a credit-risk mart. A modern lakehouse seeks to combine flexible storage with warehouse-style governance and performance.

A common analytical design separates fact tables from dimension tables. A fact table records measurable events, such as payments, with keys linking to dimensions such as customer, product, date and branch. In a star schema, the central fact table is surrounded by these dimensions. This structure supports consistent aggregation and reduces repeated descriptive data.

Figure: The business intelligence cycle from source data to governed action.

2. Preparing, Cleaning and Integrating Data

Analytical validity depends on the quality of the underlying data. Important quality dimensions include:

• Accuracy: values correctly represent the real-world event.
• Completeness: required records and fields are present.
• Consistency: the same concept has compatible values across systems.
• Timeliness: data is available at the required time.
• Validity: values conform to rules, formats and permitted ranges.
• Uniqueness: an event is not unintentionally duplicated.
• Integrity: relationships among records remain correct.

2.1 A practical preparation workflow

1. Define the decision and grain. State what one row represents: one transaction, one customer-month, one claim or one loan application. Many errors occur when transaction-level and customer-level data are mixed without aggregation.
2. Profile the data. Inspect field types, missing values, distinct values, ranges, duplicates and unusual patterns.
3. Standardise formats. Align dates, currencies, country codes, product names and identifiers. Store the original value when transformations are material.
4. Handle missing values. Determine whether a blank means unknown, not applicable, not collected or zero. Do not automatically replace missing values with zero. Imputation should be justified and documented.
5. Detect duplicates and anomalies. Use transaction IDs, timestamps and business rules. An unusually large payment may be a genuine event rather than an error.
6. Validate relationships. A payment should link to a valid account; a claim should link to a policy; an account should have a permitted product code.
7. Integrate sources. Establish common keys and definitions. Resolve conflicting customer identifiers through a controlled master-data process.
8. Document lineage. Record source, transformation, owner, refresh time and known limitations.
9. Test and reconcile. Compare totals with trusted operational or accounting systems before publication.

A useful reconciliation is:

The difference should be investigated, not simply hidden. If the two totals use different populations or cut-off times, a non-zero difference may be explainable; otherwise it indicates a defect.

2.2 Integration challenges

A bank may identify a customer by a core-banking number, a mobile number and a CRM identifier. These identifiers do not automatically refer to the same entity. Entity resolution links records that represent the same customer, while avoiding false matches. Integration also requires attention to time zones, slowly changing customer attributes, exchange rates, product hierarchies and historical restatements.

For financial data, currency conversion must specify the exchange-rate type and date. A revenue figure converted using a closing rate is not necessarily comparable with one converted using a transaction-date rate. Analysts must preserve the original currency and conversion assumptions.

3. Types of Analytics

The four commonly used categories form a progression, although real projects often combine them.

3.1 Descriptive analytics

Descriptive analytics summarises historical or current data using counts, totals, averages, ratios, distributions and trends. A mean is calculated as:

Here, xᵢ is observation i, n is the number of observations and x̄ is the arithmetic mean. The formula assumes observations are measured on a scale for which averaging is meaningful. In skewed transaction data, the median may represent a typical observation better than the mean.

A ratio must have a clearly defined numerator, denominator and population. For example, a card-approval rate is:

It is misleading to use all website visitors in the denominator if only completed applications were eligible for approval.

3.2 Diagnostic analytics

Diagnostic analytics investigates causes and contributing factors. Techniques include drill-down, segmentation, cohort comparison, variance analysis, correlation analysis and controlled testing. Correlation can identify association but does not, by itself, prove causation. A rise in digital fraud and a rise in customer complaints may both be caused by a third factor, such as a system migration.

Variance analysis separates actual performance from a reference value:

For a percentage variance, the usual denominator is the budget or other agreed baseline:

This percentage is undefined when the budget is zero and can be unstable when the denominator is very small.

3.3 Predictive analytics

Predictive analytics estimates future or unknown outcomes using historical data. Examples include credit-risk scores, claim-frequency forecasts, churn prediction and cash-flow forecasts. A binary classification model may estimate a probability between 0 and 1, but a probability is not a certainty. Decisions require a threshold, costs, benefits and governance.

A simple linear model is:

Here, y is the outcome, xⱼ are explanatory variables, β₀ is the intercept, βⱼ are coefficients and ε is the unexplained error. The model is useful only when its assumptions, data quality, functional form and validation performance are appropriate. It should not be interpreted as proof that changing xⱼ will cause a change in y.

Model assessment should use measures appropriate to the task. For binary classification, a confusion matrix distinguishes true positives, false positives, true negatives and false negatives. Precision is:

Recall is:

Increasing one may reduce the other, so the preferred balance depends on the business cost of each error.

3.4 Prescriptive analytics

Prescriptive analytics compares possible actions under objectives and constraints. For example, a collections team may prioritise accounts to contact, subject to staff capacity and customer-protection rules. A prescriptive recommendation should expose assumptions, constraints and trade-offs rather than present a black-box command.

4. KPIs, Scorecards and Dashboards

A key performance indicator (KPI) is a measure linked to a significant objective. A metric becomes a KPI only when its purpose, owner, target, time period and action implications are defined. A good KPI is relevant, clearly calculated, comparable and difficult to manipulate.

A KPI specification should include:

Common financial and operational measures include return on equity, cost-to-income ratio, non-performing-loan ratio, claims ratio, expense ratio, retention rate, service-level attainment and fraud-loss rate. Definitions must be institutionally and regulatorily consistent. For example, “profit” may mean accounting profit, operating profit or economic profit; the dashboard must identify which one is used.

A scorecard compares performance against targets across selected dimensions. A balanced scorecard commonly combines financial, customer, internal-process and learning or capability perspectives. A dashboard is an interactive presentation designed for monitoring and action. A dashboard is not simply a collection of charts: it should answer a defined set of questions for a particular audience.

4.1 Dashboard design principles

• Put the most important decision-relevant measures first.
• Use consistent definitions, scales and time periods.
• Show actuals alongside targets and prior periods.
• Use colour sparingly; do not rely on colour alone for meaning.
• Label units, currencies, dates and denominators.
• Provide drill-down without overwhelming the first view.
• Show data freshness and ownership.
• Distinguish confirmed results from forecasts.
• Include exception indicators and recommended follow-up.
• Design for accessibility, including readable contrast and text alternatives.

4.2 Choosing visualisations

• Bar charts: compare categories; use a common baseline where magnitude matters.
• Line charts: show trends over ordered time; avoid implying continuity for unrelated categories.
• Histograms: show the distribution of a numeric variable using intervals.
• Scatter plots: examine relationships between two numeric variables.
• Box plots: compare distributions, medians and possible outliers.
• Maps: show geographic patterns when location is analytically relevant.
• Tables: provide exact values and are often superior for operational checking.
• Waterfall charts: explain how components contribute to a change in a total.

Pie charts become difficult to compare when there are many categories or similar shares. Three-dimensional effects and excessive decoration distort perception. A dashboard should not conceal uncertainty, missing data or changes in definitions.

5. Applications in Banking, Finance and Insurance

Operational decisions

Operations teams use dashboards to monitor queue length, processing time, transaction failures, branch traffic, ATM availability and service-level attainment. Diagnostic analysis can separate delays caused by staffing, system latency, documentation errors or approval escalations.

Customer decisions

Customer analytics supports segmentation, product recommendation, retention and service improvement. A segment should be actionable and stable enough to support a decision. A high-value customer segment may require profitability analysis rather than simply ranking customers by balance, because servicing cost, funding cost and risk also matter.

Risk decisions

Risk analytics supports underwriting, portfolio monitoring, fraud detection, stress testing and early-warning systems. Analysts should monitor model drift, false alarms, changing economic conditions and the treatment of vulnerable customers. A risk score must not become an unreviewable substitute for professional judgement.

Financial decisions

Management reporting compares revenue, cost, margin, capital use, liquidity and budget performance. Measures should reconcile with accounting records and specify whether figures are gross or net, nominal or real, booked or settled, and actual or estimated.

Insurance decisions

Insurers analyse quote conversion, policy persistency, claim frequency, claim severity, settlement time, fraud indicators and loss ratios. A basic claims ratio can be expressed as:

The numerator and denominator must use the same period, product scope and accounting basis. The measure alone does not explain whether performance changed because of claim frequency, claim severity, pricing, reserving or exposure mix.

6. Illustrative Case: Meridian Shield Bank

Background. Meridian Shield Bank, a fictional retail bank, observed that personal-loan applications increased, while customer complaints about slow decisions also rose. Management suspected staffing shortages but lacked evidence.

Problem. Data came from the application platform, branch system, contact centre and workforce schedule. Customer IDs differed across systems, and “decision time” was calculated inconsistently.

Analysis. The analytics team defined the grain as one completed application. It retained application timestamp, decision timestamp, channel, product, branch, documentation status, escalation flag and customer segment. Records were deduplicated using the application ID. A data dictionary defined decision time as elapsed time from a complete application to a final decision. The team reconciled monthly application counts with the loan-origination system.

Descriptive analysis showed that overall median decision time increased from 18 to 31 hours. Diagnostic drill-down found that incomplete applications and manual escalations accounted for most of the increase, while staffing explained only a smaller share. Applications submitted through a new digital channel had a higher incomplete-document rate.

Approach and solution. The bank created an operational dashboard with application volume, completion rate, median decision time, 90th-percentile decision time, escalation rate and complaints per 1,000 applications. It added a diagnostic view by channel and documentation type. A rules-based notification prompted applicants to correct missing documents before submission. A predictive model was tested to identify applications likely to require manual review, but it was independently validated and subjected to fairness and explainability checks before limited use.

Outcome. In this illustrative scenario, the bank reduced avoidable rework and improved decision-time visibility. Management also learned that a single average concealed operational bottlenecks. The result was not attributed solely to the dashboard: process changes, data definitions and ownership were necessary.

Learning points. Define the population before calculating a KPI; reconcile analytical totals; distinguish symptoms from causes; involve process owners; validate models; and connect every dashboard measure to a possible action.

7. Governance, Ethics and Limitations

Data governance assigns accountability for definitions, access, quality, retention and use. A governance framework should include data owners, custodians, classification rules, lineage, access controls, incident procedures and change management.

Privacy requires lawful and fair processing, purpose limitation, data minimisation, appropriate retention and protection against unauthorised disclosure. Sensitive financial data should be encrypted in transit and at rest, with role-based access and audit logs. Data used for analysis should be pseudonymised where possible, although pseudonymisation does not eliminate re-identification risk.

Analytics can reproduce historical discrimination. A model may appear accurate overall while producing different error rates across groups. Fairness assessment should consider the legal and organisational context, relevant protected characteristics, data limitations and the consequences of errors. Removing a sensitive variable does not necessarily remove bias because other variables may act as proxies.

Important limitations include:

• Selection bias: observed data may exclude relevant cases.
• Survivorship bias: failed or exited customers may disappear from the sample.
• Measurement bias: a proxy may not measure the intended concept.
• Data drift: relationships and populations change over time.
• Confounding: a third factor may explain an apparent relationship.
• Overfitting: a model may perform well on training data but poorly on new data.
• Automation bias: users may accept a system recommendation without adequate review.
• Metric gaming: employees may optimise a target while harming the broader objective.

A sound control environment uses independent validation, access management, version control, reproducible transformations, monitoring, human review and documented escalation. Regulatory obligations vary by jurisdiction, but financial institutions generally face heightened expectations concerning consumer protection, record keeping, model risk, operational resilience and confidentiality.

8. Contemporary Developments and Best Practice

Cloud analytics, streaming data, self-service BI and machine learning have expanded analytical capability. Generative AI can assist with query writing, explanation and natural-language interfaces, but outputs require verification. It can hallucinate definitions, expose confidential information or produce biased interpretations. Institutions should define approved uses, prohibit entry of sensitive data into unauthorised tools, retain human accountability and test outputs.

A mature analytics function follows these practices:

1. Begin with a decision and an accountable owner, not a tool.
2. Establish a governed data dictionary and common business definitions.
3. Build quality checks into pipelines rather than correcting errors only at the end.
4. Separate exploratory analysis from official reporting.
5. Use appropriate baselines, denominators and comparison periods.
6. Communicate uncertainty, assumptions and data freshness.
7. Validate models on out-of-sample data and monitor performance after deployment.
8. Test accessibility, privacy, security and fairness.
9. Measure whether analysis improved an outcome, not merely whether a dashboard was viewed.
10. Retire unused KPIs and dashboards to reduce clutter and control risk.

Chapter Summary

Business intelligence converts governed data into information that supports decisions. Effective analysis begins with a clear decision, a defined population and a documented data grain. Data must be profiled, cleaned, integrated, reconciled and protected before visualisation. Descriptive analytics reports what happened; diagnostic analytics examines why; predictive analytics estimates what may happen; and prescriptive analytics compares actions under constraints. KPIs require clear definitions, owners, targets and response rules. Scorecards compare performance across objectives, while dashboards present decision-relevant information to a specific audience. Banking, finance and insurance applications include operations, customer management, credit and fraud risk, financial control and claims analysis. Ethical use requires privacy, fairness, security, explainability and human accountability. Technology increases capability but does not replace judgement, governance or professional responsibility.

Key Terms

Business intelligence: Organised use of data, technology and analysis to support business decisions.

Business analytics: Methods for describing, explaining, predicting and prescribing business outcomes.

Data warehouse: Structured repository optimised for governed analytical reporting.

Data lake: Storage environment that retains large volumes of data in varied formats.

Data mart: Analytical store focused on a particular subject or function.

Data grain: The precise meaning of one row in a dataset.

Data lineage: Record of data origin, transformations, movement and ownership.

Data quality: Fitness of data for its intended use across dimensions such as accuracy and completeness.

Descriptive analytics: Analysis of historical or current events.

Diagnostic analytics: Analysis of causes and contributing factors.

Predictive analytics: Estimation of future or unknown outcomes.

Prescriptive analytics: Evaluation or recommendation of actions under objectives and constraints.

KPI: Key measure linked to an important organisational objective.

Scorecard: Structured comparison of performance against targets across selected dimensions.

Dashboard: Interactive display designed to monitor measures and support action.

Entity resolution: Process of determining which records represent the same real-world entity.

Model drift: Decline or change in model performance as data or relationships change.

Data governance: Policies, roles and controls governing data quality, access, use and accountability.

Review Questions

Short Answer Questions

1. Define business intelligence and explain how it differs from data visualisation.
2. Why must the grain of a dataset be specified before analysis?
3. Distinguish descriptive analytics from diagnostic analytics.
4. Identify four dimensions of data quality.
5. What information should be included in a KPI specification?

Long Answer Questions

1. Explain a complete workflow for preparing and integrating banking data, including reconciliation and lineage.
2. Compare descriptive, diagnostic, predictive and prescriptive analytics, using one financial-services example for each.
3. Discuss the principles of effective dashboard design for a bank’s operations manager.
4. Explain how data quality, model validation and governance influence credit-risk analytics.
5. Analyse the ethical and privacy risks of using customer data for insurance or banking decisions.

Analytical / Application Questions

1. A bank reports that its approval rate increased from 60% to 75%, but applications fell sharply. What additional denominators, segments and operational measures should be examined before concluding that performance improved?
2. An insurer’s claims ratio increased while the number of claims remained stable. Develop a diagnostic analysis to determine whether the change resulted from severity, earned premium, product mix, reserving or data-definition changes.
3. Design a dashboard for a digital-loan process. Specify five KPIs, their definitions, suitable visualisations, target or comparison basis, owners and the actions that should follow adverse results.

Multiple Choice Questions with Answers

Question 1

Which analytics category primarily answers the question “What happened?”

A. Descriptive analytics

B. Diagnostic analytics

C. Predictive analytics

D. Prescriptive analytics

Correct Answer: A. Descriptive analytics

Question 2

What is the most important reason for defining data grain before combining datasets?

A. To increase the number of charts

B. To ensure that each row has a clear and consistent meaning

C. To eliminate the need for data validation

D. To guarantee that all values are normally distributed

Correct Answer: B. To ensure that each row has a clear and consistent meaning

Question 3

Which visualisation is generally most suitable for showing a monthly trend in claim volume?

A. Line chart

B. Pie chart with three-dimensional effects

C. Unordered text list

D. Single-value gauge only

Correct Answer: A. Line chart

Question 4

A report has actual revenue of $108,000 and a budget of $120,000. What is the percentage variance using the budget as denominator?

A. −10%

B. 10%

C. −12%

D. 12%

Correct Answer: A. −10%

Question 5

Which measure is calculated as true positives divided by true positives plus false positives?

A. Recall

B. Precision

C. Accuracy gap

D. Specificity

Correct Answer: B. Precision

Question 6

Which statement about correlation is correct?

A. Correlation always proves causation

B. Correlation measures association but does not by itself prove causation

C. Correlation can be calculated only for categorical variables

D. Correlation eliminates the need for diagnostic analysis

Correct Answer: B. Correlation measures association but does not by itself prove causation

Question 7

Which item is essential in a well-defined KPI?

A. An unspecified data source

B. A target, owner and calculation definition

C. A decorative colour scheme

D. A guarantee that the value will improve

Correct Answer: B. A target, owner and calculation definition

Question 8

For the claims ratio formula, what is the usual denominator?

A. Total number of employees

B. Earned premium

C. Number of policies quoted

D. Total assets

Correct Answer: B. Earned premium

Question 9

Which is an example of prescriptive analytics?

A. Reporting last month’s branch transactions

B. Listing customers by current balance

C. Estimating next month’s claim frequency

D. Selecting collection actions subject to staffing and risk constraints

Correct Answer: D. Selecting collection actions subject to staffing and risk constraints

Question 10

Why can removing a sensitive attribute fail to eliminate bias from a predictive model?

A. Models cannot use numerical data

B. Other variables may act as proxies for that attribute

C. Dashboards automatically reintroduce the attribute

D. Data cleaning always creates bias

Correct Answer: B. Other variables may act as proxies for that attribute