Cloud, Lakehouse and Real-Time Warehousing

Introduction

Data warehousing has evolved from a centralized relational database into a distributed analytical ecosystem. Traditional warehouses generally combined storage, computation, workload management, and governance in one platform. Cloud-native architectures separate these responsibilities, use elastic infrastructure, and integrate batch, streaming, semi-structured, and machine-generated data. The resulting environment may include a cloud data warehouse, a data lake, an open table format, a stream-processing system, and real-time serving layers.

This evolution is driven by several pressures: rapidly increasing data volumes, unpredictable analytical workloads, demand for near-real-time decisions, dependence on multiple cloud and software platforms, and the need to control infrastructure costs. The data lakehouse attempts to combine the low-cost and flexible storage of a data lake with the reliability, performance, and governance associated with a warehouse. Real-time warehousing extends this objective by making newly generated data available for analysis within seconds or minutes.

The central design problem is not simply selecting a product. It is determining how storage, computation, data formats, ingestion, governance, and serving should be composed to satisfy business requirements for latency, scalability, interoperability, reliability, security, and cost.

Learning Objectives

After studying this chapter, the learner should be able to:

1. Explain the architecture and operating principles of cloud-native data warehouses.

2. Evaluate separation of storage and compute and distinguish elastic from serverless execution.

3. Compare data lakes, data warehouses, and lakehouses.

4. Explain the role of open table formats in transactional data lakes.

5. Design batch and streaming ingestion pipelines for real-time analytics.

6. Assess architectures using scalability, latency, interoperability, governance, and cost criteria.

7. Identify operational, ethical, regulatory, and vendor-related risks.

8. Recommend an architecture for a realistic organizational scenario.

1. From Enterprise Warehouses to Cloud-Native Analytics

A conventional enterprise data warehouse usually stores curated, structured data in a relational schema. Extract-transform-load (ETL) processes load data from operational systems into staging and warehouse layers. Compute capacity is provisioned in advance, and storage and processing are commonly scaled together.

Cloud-native warehousing changes this model in four important ways:

• Object storage as a durable foundation: Data is stored in services such as cloud object stores, which provide high durability and large capacity at comparatively low storage cost.
• Independent compute: Query engines can be scaled, paused, or replaced without moving the underlying data.
• Managed services: The provider operates servers, storage hardware, patching, and much of the failure recovery.
• Elastic consumption: Capacity can increase during demand peaks and decrease when workloads decline.

A cloud warehouse may still present a familiar SQL interface while internally distributing query planning, execution, caching, metadata management, and storage across many machines. It can also support structured tables, semi-structured documents, external data, and machine-learning workloads.

Cloud-native does not mean that all architecture decisions are automatically optimal. A managed service can reduce operational effort but increase dependence on a provider. Elasticity can control costs when workloads vary, but poorly governed auto-scaling can create unexpected expenditure. Therefore, cloud architecture must be evaluated as an economic and organizational system as well as a technical one.

2. Separation of Storage and Compute

2.1 Concept

Separation of storage and compute means that persistent data and query-processing resources are managed as distinct architectural layers. Storage is usually implemented on durable object storage or a warehouse-managed equivalent. Compute consists of clusters, virtual warehouses, SQL engines, or serverless execution resources that read the stored data.

In a coupled system, increasing query capacity may require purchasing additional storage, even when storage is not the limiting resource. In a decoupled system, an organization can add compute for a heavy workload without duplicating all data. Conversely, storage can grow while compute remains unchanged.

The model supports:

• independent scaling;
• multiple compute groups for different workloads;
• workload isolation;
• temporary or on-demand processing;
• shared data access by different analytical engines;
• suspension of idle compute.

2.2 Workload isolation

A single analytical environment may support executive dashboards, data science, scheduled transformation, ad hoc exploration, and data ingestion. These workloads have different resource profiles. A dashboard requires predictable response time, whereas a machine-learning job may require large scans and long execution time.

Separate compute pools or virtual warehouses can reduce interference. However, isolation does not eliminate contention entirely. Shared metadata services, object storage bandwidth, concurrency limits, network capacity, and platform quotas can remain bottlenecks.

2.3 Cost model

A simplified total-cost model for a cloud analytical workload is:

\[

C_{total}=C_{storage}+C_{compute}+C_{ingress/egress}+C_{services}+C_{operations}

\]

where each term represents the relevant billing category over a specified period. The model is a planning identity, not a universal provider pricing formula. Actual charges may include per-query processing, reserved capacity, data-transfer tiers, API requests, replication, backup, and support.

For compute billed by time, a basic estimate is:

\[

C_{compute}=r_h \times h

\]

where \(r_h\) is the price per compute-hour and \(h\) is the number of billed compute-hours. If a cluster has \(n\) equivalent nodes running for \(t\) hours at a node-hour rate \(r_n\), then \(C_{compute}=ntr_n\), assuming linear pricing and no minimum-duration or discount effects.

For example, a four-node cluster used for 6 hours at $0.80 per node-hour costs:

\[

4\times6\times0.80=19.20

\]

This excludes storage, data transfer, and other charges. The example illustrates why workload scheduling and automatic suspension matter.

2.4 Benefits and limitations

The major benefits are elasticity, workload isolation, and better alignment between resource use and demand. Limitations include platform-specific behavior, cold-start delays, concurrency controls, data-transfer charges, and the possibility that repeated scans make compute costs exceed those of a more optimized design. Query optimization, partition pruning, clustering, caching, materialized views, and appropriate file sizing remain important in a decoupled architecture.

3. Serverless Analytical Services

Serverless analytics abstracts infrastructure provisioning from the user. A user submits SQL, a job, or a streaming query, and the provider allocates resources automatically. “Serverless” does not mean that no servers exist; it means that customers do not manage the servers directly.

Serverless execution is especially useful when workloads are intermittent, difficult to forecast, or operationally expensive to administer. It can support ad hoc queries, event-driven transformations, and short-lived processing tasks. It may be less suitable for continuously active, latency-sensitive workloads if startup delays, concurrency limits, or per-request pricing are significant.

Important evaluation dimensions include:

• Startup latency: time required before computation begins;
• Execution latency: time taken after resources are available;
• Concurrency: number of simultaneous jobs supported;
• Predictability: stability of performance and cost;
• Control: ability to tune memory, networking, locality, and execution plans;
• Portability: ease of moving jobs to another platform.

A serverless query service can coexist with provisioned warehouses. A common pattern is to use serverless execution for occasional exploration and a reserved or elastic cluster for predictable dashboards and recurring transformations.

4. Data Lakes, Warehouses, and Lakehouses

4.1 Data warehouse

A data warehouse emphasizes governed, structured, integrated data and optimized SQL analytics. It commonly provides schema management, transactions, workload management, security controls, and business-oriented dimensional or relational models. Its limitations may include storage cost, restricted data types, and dependence on proprietary formats or execution engines.

4.2 Data lake

A data lake stores structured, semi-structured, and unstructured data, often in open or inexpensive object storage. It supports exploration and preserves raw data for future uses. A basic lake, however, may suffer from schema drift, incomplete metadata, duplicate records, weak transaction support, difficult updates, and unreliable concurrent access. Such an environment is sometimes described as a “data swamp” when discoverability and governance deteriorate.

4.3 Lakehouse

A lakehouse is an architectural approach that adds warehouse-like management to data-lake storage. Its intended properties include:

• open, durable storage;
• tabular abstractions over files;
• atomic commits and consistent reads;
• schema enforcement and evolution;
• time travel or historical version access;
• batch and streaming support;
• interoperability across processing engines;
• governance and performance optimization.

The lakehouse is not a single product. It is a combination of storage, metadata, table-management, query, governance, and processing technologies. Its quality depends on how these components work together.

5. Open Table Formats

Object stores contain files, not database tables. An open table format provides metadata and protocols that make a collection of files behave like a managed table. Widely used examples include Apache Iceberg, Delta Lake, and Apache Hudi. Their detailed features differ, but common capabilities include snapshots, schema management, partition metadata, and controlled commits.

5.1 Metadata and snapshots

A table format records which data files belong to a table version. A reader uses the current metadata snapshot rather than scanning every file blindly. A write creates new files and updates metadata. If the metadata commit succeeds, readers see the new table state; if it fails, the prior state remains available. This supports atomicity at the table-commit level.

5.2 Schema evolution

Schema evolution permits controlled changes such as adding columns, renaming fields, or widening compatible types, depending on the format and engine. It is not equivalent to accepting any incoming schema. Strong governance requires compatibility rules, ownership, validation, and documentation.

5.3 Partitioning and file organization

Partitioning divides data according to values such as date or region. It can reduce scanning when query predicates align with the partition design. Excessive partitioning creates many small files and metadata overhead. A modern design therefore combines logical partitioning with file compaction, clustering, sorting, or data skipping.

5.4 Interoperability and its limits

Open formats improve interoperability because multiple engines can read the same table representation. However, interoperability depends on compatible implementations, catalog services, table features, transaction protocols, encryption, and data types. “Open format” does not guarantee that every engine supports every feature. Organizations should test actual read, write, update, and streaming behavior rather than rely only on marketing claims.

6. Streaming Ingestion and Real-Time Warehousing

6.1 Event-driven data

Streaming data is produced continuously by applications, sensors, transactions, logs, clickstreams, and operational databases. A streaming architecture commonly contains producers, a durable event broker, stream-processing jobs, a storage or table sink, and a serving layer.

Important event properties include:

• event time: when the event occurred;
• processing time: when a system processed it;
• ingestion time: when it entered the platform;
• event key: an identifier used for partitioning or correlation;
• sequence or offset: a position used for tracking progress;
• schema and version: the structure and compatibility contract.

Confusing event time with processing time can produce incorrect windows, late-arrival handling, and business metrics.

6.2 Delivery semantics

Streaming systems are often described using three delivery models:

• At-most-once: an event may be lost, but is not normally delivered more than once.
• At-least-once: an event is not intentionally lost after successful acknowledgment, but duplicates may occur.
• Exactly-once: the end-to-end result is equivalent to processing each event once under defined failure assumptions.

Exactly-once is not a magical property of a broker alone. It requires coordinated source offsets, processing state, sink commits, retries, and transactional or idempotent behavior. Many practical systems use at-least-once delivery plus idempotent writes and deduplication.

An idempotent sink produces the same final state when the same event is applied repeatedly. This can be achieved using a stable event identifier, merge semantics, uniqueness constraints where supported, or a stateful deduplication window.

6.3 Event-time windows

A tumbling window divides time into non-overlapping intervals. A five-minute tumbling window might calculate sales for 10:00–10:05, 10:05–10:10, and so on. A sliding window can overlap, while a session window groups events separated by less than a specified inactivity gap.

Late data is handled using watermarks, allowed lateness, correction records, or reprocessing. A watermark is an operational estimate that events earlier than a particular event-time boundary are unlikely to arrive. It is not proof that no later event will appear.

6.4 Latency measures

Real-time systems should distinguish:

• ingestion latency: source to platform;
• processing latency: event arrival to transformation completion;
• serving latency: completion to query visibility;
• end-to-end freshness: source event to business-user availability;
• query latency: time required to answer a user query.

A system can have fast ingestion but poor freshness if transformations or serving updates are delayed. Service-level objectives should specify percentile latency, not only an average. For example, a requirement might state that 99% of events become queryable within 30 seconds.

7. Real-Time Analytical Architectures

A real-time warehouse may use one of several patterns.

7.1 Lambda-style architecture

A batch layer computes complete historical results, while a speed layer provides rapid but possibly temporary results. A serving layer combines them. This can tolerate different processing paths but creates duplicated logic and reconciliation complexity.

7.2 Kappa-style architecture

A streaming pipeline is treated as the primary processing path. Historical results can be reconstructed by replaying retained events. This reduces duplicated logic but requires durable event retention, replayable transformations, and adequate reprocessing capacity.

7.3 Unified incremental architecture

A modern lakehouse may use one table and transaction model for both streaming writes and batch reads. Incremental processing updates curated tables as new data arrives, while periodic compaction and quality checks maintain performance. This approach can simplify governance but still requires careful handling of late data, updates, deletes, and concurrent writers.

7.4 Serving choices

Analytical results may be queried directly from a warehouse or lakehouse, served through materialized views, or copied into a low-latency operational analytical store. Direct querying simplifies data movement but may not satisfy sub-second interaction requirements. A specialized serving store improves response time for known access patterns but introduces synchronization and duplication.

8. Evaluation Framework: Scalability, Latency, Interoperability, and Cost

Architecture selection should begin with measurable requirements rather than technology preference.

Scalability

Scalability concerns data volume, ingestion rate, concurrent users, query complexity, and workload growth. Vertical scaling increases resources in an existing execution unit; horizontal scaling adds execution units. Elastic systems may scale automatically, but scaling speed and maximum capacity must be tested. Storage scalability is not equivalent to query scalability.

Latency

Latency requirements should be classified as batch, hourly, minute-level, sub-minute, or interactive. Low latency often increases cost and operational complexity because it requires continuous compute, efficient indexing or clustering, small state updates, and careful backpressure management.

Interoperability

Assess support for SQL, APIs, open formats, catalog standards, identity systems, orchestration tools, and multiple processing engines. Consider both syntactic compatibility and semantic compatibility. A tool may read files successfully while interpreting timestamps, nulls, decimal precision, or deletes differently.

Cost

Evaluate total cost of ownership, not only storage price. Include compute, data transfer, replication, observability, support, engineering labor, migration, compliance, and failure recovery. Cost should be measured per useful outcome, such as cost per million events processed or cost per dashboard query, while recognizing that such measures depend on workload and quality requirements.

Governance and reliability

A platform that is inexpensive but produces inaccurate, unauditable, or insecure results is not economically efficient. Evaluation should include lineage, access control, retention, recovery point objectives, recovery time objectives, data-quality monitoring, and auditability.

9. Worked Architecture Example

A retailer receives 50,000 point-of-sale events per minute during peak periods and requires fraud alerts within 20 seconds. It also needs daily financial reporting and multi-year historical analysis.

A suitable conceptual design is:

1. Point-of-sale systems publish immutable events with unique transaction identifiers and event timestamps.

2. A durable event broker partitions events by store or transaction key.

3. A stream processor validates schemas, removes duplicates, enriches events with reference data, and writes a bronze table.

4. A curated silver table applies business rules and handles corrections.

5. A gold layer contains fraud features, store metrics, and finance-ready aggregates.

6. A low-latency serving view supports fraud operations, while the warehouse supports broader analysis.

7. Raw events are retained sufficiently long to permit replay and audit.

8. Batch reconciliation compares event-derived totals with the finance system.

The architecture should monitor event lag, processing failures, duplicate rates, late-arrival rates, table freshness, query latency, and cost. The 20-second objective must be tested at a high percentile, such as the 99th percentile, under peak load. A median of 5 seconds is insufficient if 5% of events take several minutes.

10. Illustrative Case: Metropolitan Transit Analytics

Background

The following is an illustrative case. A metropolitan transit authority operates buses and trains across a large region. Vehicle telemetry, ticket validations, maintenance records, and service alerts are generated continuously. Managers need current congestion information, while planners require years of historical data.

Problem

The authority’s traditional warehouse receives nightly extracts. This produces reliable financial reports but cannot identify service disruptions quickly. Data from the ticketing system, vehicle sensors, and maintenance application also uses inconsistent identifiers and schemas.

Analysis

The requirements are heterogeneous. Operations require minute-level freshness; planning requires large historical scans; maintenance analysts need semi-structured diagnostic data; regulators require traceable records. A single database configuration would either be too expensive for all workloads or too slow for operational alerts.

Approach

The authority adopts an event broker for telemetry and ticket events, a schema registry for version control, and an object-store lakehouse using an open table format. Raw events are retained in an immutable layer. Streaming jobs create validated and enriched tables, while scheduled batch jobs reconcile totals and compact small files. Separate compute resources serve operational dashboards, data science, and finance reporting.

Data contracts define mandatory identifiers, event-time semantics, allowable changes, and ownership. A catalog records lineage and sensitivity. Access policies restrict personally identifiable travel information, and retention rules delete or anonymize data when the lawful purpose expires.

Outcome

Operations staff receive disruption indicators within minutes rather than the next day. Analysts can query current and historical data through consistent tables. The authority reduces duplicate extraction pipelines, but it must continue funding data-quality monitoring, platform engineering, and cost controls. The case demonstrates that a lakehouse is not merely a storage migration; it is an organizational change involving contracts, governance, skills, and operating procedures.

Learning points

• Different latency requirements justify separated compute and serving paths.
• Open table formats improve reuse but do not solve inconsistent business definitions.
• Streaming and batch reconciliation are complementary.
• Privacy controls must be designed into event and analytical layers.
• Operational success requires monitoring and ownership, not only technology.

11. Governance, Security, Ethics, and Regulation

Cloud and real-time architectures increase the speed and breadth of data use, which can amplify harm as well as value. Security controls should include encryption in transit and at rest, centralized identity, least-privilege access, network segmentation, secret management, audit logging, and controlled administrative access.

Governance must address data ownership, retention, lineage, classification, quality, and permissible use. Streaming systems require special attention because events may contain identifiers, location traces, behavioral profiles, or sensitive operational details. Real-time availability can make inappropriate access or automated decisions occur before human review.

Organizations should consider privacy principles such as purpose limitation, data minimization, transparency, access rights, correction, deletion where applicable, and lawful retention. Regulatory obligations vary by jurisdiction and sector; therefore, architecture teams should involve legal, compliance, security, and records-management specialists rather than treating compliance as a final deployment checklist.

Automated analytics and AI can introduce bias, proxy discrimination, and opaque decision-making. Feature stores and real-time scoring pipelines should document data provenance, model versions, threshold changes, and human escalation procedures. Accuracy alone is not sufficient: fairness, explainability, robustness, and contestability may be required.

12. Contemporary Developments and Best Practices

Current developments include streaming lakehouses, open catalogs, managed table services, vector and geospatial analytics, real-time machine-learning features, data mesh operating models, and increasingly autonomous optimization. These developments should be assessed critically. A new feature is valuable only if it improves an identified requirement without unacceptable lock-in, risk, or complexity.

Recommended practices are:

1. Define freshness, availability, recovery, and cost objectives before selecting products.

2. Use immutable raw data and reproducible transformations where auditability matters.

3. Establish data contracts and schema-compatibility policies.

4. Design for retries, duplicate events, late data, replay, and partial failure.

5. Separate compute by workload when contention affects service objectives.

6. Control small files through compaction and review partition strategies regularly.

7. Measure percentile latency, freshness, quality, and cost continuously.

8. Test interoperability with real writes, updates, deletes, and schema changes.

9. Apply security and privacy controls at ingestion, storage, processing, and serving layers.

10. Maintain exit plans, documented dependencies, and portable representations where vendor risk is material.

Challenges and Limitations

Cloud-native warehousing does not remove data-engineering complexity. Distributed systems fail partially, schemas evolve, network links break, and business definitions conflict. Eventual consistency in some services can complicate validation. Continuous processing can be more expensive than scheduled batch processing. Open formats improve portability but may expose differences among engines. Managed services reduce administration but can obscure system behavior and make troubleshooting difficult.

A further limitation is organizational: real-time data requires owners who can define event meaning, service objectives, quality thresholds, and incident responses. Without these responsibilities, a technically advanced platform may produce fast but unreliable information.

Chapter Summary

Cloud-native warehousing separates durable storage from elastic computation and often combines managed, serverless, and provisioned services. This separation enables workload isolation and flexible scaling, but cost, concurrency, transfer, and performance must be measured rather than assumed. Data lakes provide flexible low-cost storage, while lakehouses add table management, transactions, schema control, and analytical usability. Open table formats improve interoperability and historical management, but compatibility depends on actual engine and catalog support.

Real-time warehousing uses event brokers, stream processors, transactional or idempotent sinks, incremental tables, and serving layers. Event time, processing time, delivery semantics, late data, watermarks, and percentile latency are essential concepts. Lambda, Kappa, and unified incremental architectures represent different trade-offs in complexity, replay, and operational behavior.

Architecture decisions should be evaluated through scalability, latency, interoperability, cost, governance, security, and reliability. The most effective solutions combine technical design with data contracts, monitoring, privacy controls, reconciliation, and clear ownership. Cloud, lakehouse, and real-time technologies are means to meet analytical objectives, not substitutes for sound data management.

Key Terms

Cloud-native warehouse: A warehouse designed around managed cloud infrastructure, elasticity, and distributed services.

Separation of storage and compute: Independent management and scaling of persistent data and processing resources.

Serverless analytics: Execution in which the provider allocates infrastructure without direct server administration.

Data lake: Flexible storage for structured, semi-structured, and unstructured data, commonly on object storage.

Lakehouse: An architecture combining data-lake storage with warehouse-like table management and governance.

Open table format: Metadata and transaction conventions that make files behave as a managed analytical table.

Snapshot: A consistent version of a table’s metadata and referenced data files.

Schema evolution: Controlled modification of table structure over time.

Partitioning: Organizing data into subsets to reduce unnecessary scanning.

Compaction: Combining small files into larger files to improve read efficiency.

Event time: The time at which a business or system event occurred.

Processing time: The time at which a processing system handled an event.

Watermark: An estimate of the event-time boundary below which late events are unlikely.

At-least-once delivery: Delivery semantics that may produce duplicates but aim to avoid loss after acknowledgment.

Idempotence: A property in which repeated application of the same operation produces the same final result.

Freshness: The age of the newest data available to a user or application.

Workload isolation: Separating resources or execution groups to reduce interference among workloads.

Data contract: An explicit agreement about event structure, meaning, ownership, quality, and compatibility.

Data lineage: Information describing the origin, transformations, and destinations of data.

Review Questions

Short Answer Questions

1. What is meant by separation of storage and compute?

2. Distinguish a data lake from a lakehouse.

3. Why are event time and processing time different?

4. What problem does an open table format address?

5. Define idempotent processing.

Long Answer Questions

1. Explain the benefits and limitations of serverless analytical services.

2. Compare Lambda-style, Kappa-style, and unified incremental architectures.

3. Discuss how partitioning, compaction, and snapshots affect lakehouse performance and reliability.

4. Develop a framework for evaluating a cloud warehouse using scalability, latency, interoperability, and cost.

5. Explain the governance, privacy, and security requirements of a real-time warehouse.

Analytical / Application Questions

1. An organization requires dashboards within 15 seconds, monthly reporting, and ten years of historical analysis. Propose a multi-layer architecture and justify the storage, compute, ingestion, and serving choices.

2. A streaming pipeline processes events at least once and produces duplicate sales records during retries. Design a method for deduplication and explain the assumptions needed for correctness.

3. Two platforms offer equivalent SQL functionality: Platform A has lower storage cost but expensive data egress; Platform B costs more for storage but supports several engines through an open table format. Recommend an evaluation method rather than an immediate choice, identifying the workload measurements required.

Multiple Choice Questions with Answers

Question 1

What is the principal architectural benefit of separating storage and compute?

A. It eliminates the need for metadata

B. It permits storage and processing capacity to scale independently

C. It guarantees sub-second query latency

D. It removes all data-transfer charges

Correct Answer: B. It permits storage and processing capacity to scale independently

Question 2

Which characteristic most directly distinguishes a lakehouse from a basic data lake?

A. The lakehouse stores only relational data

B. The lakehouse cannot use object storage

C. The lakehouse adds managed tables, transactions, and governance capabilities

D. The lakehouse requires all data to be streamed

Correct Answer: C. The lakehouse adds managed tables, transactions, and governance capabilities

Question 3

Which statement about at-least-once delivery is correct?

A. It guarantees that every event is processed exactly one time

B. It may produce duplicates and therefore often requires idempotence or deduplication

C. It intentionally discards all late events

D. It prevents replay of historical events

Correct Answer: B. It may produce duplicates and therefore often requires idempotence or deduplication

Question 4

What is the purpose of a watermark in event-time processing?

A. To encrypt events before storage

B. To estimate the progress of event time and manage late data

C. To increase object-store capacity

D. To assign a permanent customer identifier

Correct Answer: B. To estimate the progress of event time and manage late data

Question 5

A four-node cluster runs for 6 hours at $0.80 per node-hour. Ignoring other charges, what is the compute cost?

A. $4.80

B. $10.80

C. $19.20

D. $24.00

Correct Answer: C. $19.20

Question 6

Which practice best supports reliable replay of streaming data?

A. Discarding raw events immediately after transformation

B. Retaining immutable events with offsets or stable identifiers

C. Using processing time as the only timestamp

D. Disabling schema validation

Correct Answer: B. Retaining immutable events with offsets or stable identifiers

Question 7

Which metric describes the time from a source event until it is available for business queries?

A. End-to-end freshness latency

B. Storage capacity

C. Partition cardinality

D. Compute utilization only

Correct Answer: A. End-to-end freshness latency

Question 8

Why can excessive partitioning harm a lakehouse?

A. It always prevents schema evolution

B. It creates many small files and metadata overhead

C. It disables all batch processing

D. It removes table snapshots

Correct Answer: B. It creates many small files and metadata overhead

Question 9

Which statement about serverless analytics is accurate?

A. No physical servers are used

B. Users directly patch the underlying operating system

C. The provider manages resource allocation, while users may still face concurrency and startup limits

D. It is always cheaper than provisioned compute

Correct Answer: C. The provider manages resource allocation, while users may still face concurrency and startup limits

Question 10

Why is open table format support not by itself a guarantee of interoperability?

A. All open formats require proprietary hardware

B. Different engines may support different table features and data semantics

C. Open formats cannot store metadata

D. Open formats work only with unstructured text

Correct Answer: B. Different engines may support different table features and data semantics