Big Data and Cloud Analytics

Chapter 8
Streaming and Real-Time Analytics

Post Graduate Diploma in Data Science
Big Data and Cloud Analytics   •   Post Graduate

8. Streaming and Real-Time Analytics

Introduction and Context

Traditional data analytics usually assumes that data can be collected, stored, cleaned, and analysed in batches. This model remains valuable for historical reporting, regulatory analysis, model training, and many forms of business intelligence. However, important decisions increasingly depend on information that is changing continuously. A payment provider must detect suspicious transactions while they occur; a logistics company must respond to vehicle-location changes; an industrial plant must identify abnormal operating conditions before equipment fails; and a digital platform must adapt recommendations as user behaviour develops.

Streaming and real-time analytics address this need by treating data as an ongoing flow of events rather than as a static collection of records. An event may represent a payment, sensor reading, click, shipment update, system error, or change in customer status. A streaming system receives these events, transports them reliably, processes them with low delay, and produces actions, alerts, metrics, or updated analytical results.

The term real time does not always mean instantaneous processing. It means that the acceptable delay is defined by the business or operational context. A fraud-detection system may require results within seconds, whereas an industrial safety system may require milliseconds. A dashboard refreshed every minute may be considered near real time, even though it would not satisfy an autonomous control application. Consequently, the design of a streaming system begins with an explicit understanding of latency, correctness, availability, cost, and business impact.

This chapter examines the architecture and reasoning required to design event-driven systems. It focuses on message brokers, stream processors, event time and processing time, windowing, state management, delivery guarantees, complex event processing, operational risks, and professional applications.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain the role of event-driven architecture, message brokers, and stream processors.

2. Distinguish event time, processing time, ingestion time, and the effects of out-of-order data.

3. Select appropriate tumbling, sliding, session, and global windows for analytical tasks.

4. Explain state management, checkpoints, watermarks, and recovery in streaming systems.

5. Compare at-most-once, at-least-once, and exactly-once delivery semantics.

6. Design basic complex event processing and event-driven analytical workflows.

7. Evaluate the technical, organizational, ethical, and governance implications of real-time analytics.

8. Apply streaming concepts to an operational case and justify design decisions.

From Data Pipelines to Event-Driven Systems

An event is a recorded occurrence that is meaningful to a system or organization. It normally contains an identifier, event type, timestamp, producer information, payload, and sometimes a schema version. For example, a payment event might contain a transaction identifier, customer identifier, amount, currency, merchant, device, and time of authorization.

Event-driven architecture separates the producer of an event from the consumers that use it. A payment service publishes a transaction event without needing to know whether the event will later be used for fraud detection, accounting, customer notification, or operational reporting. This decoupling improves extensibility and allows multiple applications to consume the same data for different purposes.

A typical streaming architecture contains four logical layers:

  • Event producers generate events from applications, databases, sensors, devices, or external services.
  • Ingestion and transport services accept, buffer, partition, replicate, and distribute events.
  • Stream processors filter, transform, enrich, aggregate, join, and analyse events continuously.
  • Sinks and actions write results to databases, data lakes, dashboards, alerting systems, machine-learning services, or automated controls.

The layers are logical rather than necessarily separate products. A single platform may provide transport, processing, state storage, and connectors. Nevertheless, distinguishing their responsibilities helps architects identify failure points and select suitable technologies.

A message broker is a durable intermediary through which producers publish messages and consumers retrieve them. Unlike a simple point-to-point queue, a modern event broker commonly supports topics, partitions, consumer groups, replication, retention, replay, and independent consumers. A topic represents a logical stream of related events. Partitions allow a topic to be processed in parallel, while preserving order within each partition. A consumer group distributes partitions among instances so that a stream can scale horizontally.

Partitioning is both a performance and a correctness decision. If events for the same customer must be processed in order, the customer identifier is a possible partition key. If the key is poorly distributed, however, one partition may become a hotspot. A system designer must therefore balance ordering requirements, load distribution, scalability, and privacy considerations.

A stream processor operates on events as they arrive. Its functions may include projection, filtering, normalization, enrichment from reference data, aggregation, joins, pattern detection, and routing. Stream processors differ from ordinary application code because they must handle unbounded input, partial results, late data, failures, replay, and continuously changing state.

The Temporal Problem: What Does “Now” Mean?

Time is one of the most important design issues in streaming analytics. Four notions are especially useful:

  • Event time is the time at which the event actually occurred according to its producer.
  • Processing time is the time at which a stream processor handles the event.
  • Ingestion time is the time at which a platform accepts the event into the streaming infrastructure.
  • Wall-clock time is the current time according to a system clock.

These times can differ substantially. A mobile device may generate an event while disconnected from a network and upload it several minutes later. A sensor may have an inaccurate clock. A network delay may cause events to arrive out of order. If a business question asks, “How many purchases occurred between 10:00 and 10:05?”, event time is usually the correct basis. If the question asks, “How many messages did the system process during this minute?”, processing time may be appropriate.

Processing based on processing time is simple and often produces low-latency results, but it can misclassify delayed events. Event-time processing produces more meaningful historical calculations, but it requires mechanisms for handling incomplete knowledge. The processor cannot immediately know that no earlier event will arrive in the future.

A watermark is an advancing indication that the system believes events earlier than a particular event-time boundary are unlikely to arrive. For example, a watermark of 10:05:00 suggests that the system is ready to finalize many calculations involving times before 10:05. Watermarks are estimates, not proof. A late event may still arrive, requiring correction, reopening of a window, or a defined late-data policy.

A system should specify an allowed lateness period. Events arriving within that period may update results; events arriving after it may be discarded, redirected to a late-data stream, or used to issue corrections. The choice depends on the cost of incorrect results, the need for finality, storage capacity, and business tolerance.

Windowing Continuous Data

Most analytics cannot aggregate an infinite stream as one unbounded calculation. Windowing divides the stream into finite logical regions. A window is defined by a type, a time or count boundary, an assignment rule, and a policy for late events.

A tumbling window consists of fixed, non-overlapping intervals. A five-minute tumbling window might calculate total sales from 10:00 to 10:05, then from 10:05 to 10:10. It is appropriate for periodic reports and operational summaries where each event belongs to one interval.

A sliding window has a fixed length but advances at a smaller interval. A ten-minute window evaluated every one minute provides a continuously updated view of the preceding ten minutes. Sliding windows are useful for rolling averages, recent error rates, and anomaly detection, although they require more computation because events can participate in multiple results.

A hopping window is often used to describe fixed-size windows that advance by a hop interval. If the window size is ten minutes and the hop is five minutes, windows overlap by five minutes. Terminology varies across platforms, so the implementation definition should always be checked.

A session window groups events separated by periods of inactivity. For example, a user session may continue while clicks occur within 30 minutes of one another and close after a longer gap. Session windows are suitable for behavioural analysis but can be computationally more complex because a late event may merge two previously separate sessions.

A count-based window contains a specified number of events rather than a time duration. It can be useful when every observation should contribute equally, although it may not correspond to a meaningful business period.

Window design must match the decision being supported. A five-minute window may be useful for detecting a sudden service outage, but unsuitable for measuring a weekly retention metric. Analysts must also define whether a result is provisional or final, how late events are handled, and whether duplicate events are removed.

State, Checkpoints, and Recovery

Streaming operations often require state. A count of transactions per customer, the last known location of a device, a running average, or an open fraud-detection pattern cannot be calculated from the current event alone. The processor must retain information across events.

State may be local to a processing instance, stored in an embedded state store, or maintained in an external database. Local state can provide high performance, while external state may simplify sharing and recovery. In distributed systems, the state must be associated consistently with the partition or key that owns it.

A checkpoint is a durable record of processing progress and, in robust systems, relevant operator state. If a processor fails, it can restore the checkpoint and resume from a known position rather than starting from the beginning. Checkpoint frequency involves a trade-off: frequent checkpoints reduce recovery work but increase overhead; infrequent checkpoints improve normal performance but may require more replay.

State also creates risks. Unbounded cardinality can cause memory growth when a system retains state for an unlimited number of users or keys. Retention periods, state expiration, compaction, and monitoring are therefore essential. A stateful processor must also define how updates are emitted. It may produce every intermediate result, only changes, or a final result after a watermark passes.

A useful design principle is to make processing as deterministic and replayable as possible. If the same input events and configuration produce the same result, failures and audits are easier to manage. External side effects, such as sending an email or charging a customer, require special care because replay may repeat the action.

Delivery Guarantees and Correctness

Delivery semantics describe what the system promises when failures, retries, or network problems occur.

At-most-once processing means an event is delivered zero or one time. It avoids duplicates but may lose events if a failure occurs before durable acknowledgement. This may be acceptable for low-value telemetry or approximate monitoring.

At-least-once processing means an event is not intentionally lost, but it may be processed more than once. This is common because a consumer may process an event and fail before acknowledging it. At-least-once systems require idempotent operations, deduplication keys, or transactional sinks.

Exactly-once processing aims to ensure that each event affects the logical result once, even if internal retries occur. Exactly-once is not a universal property of an entire organization. It depends on the boundaries covered by transactions, the broker, the processor, and the destination. An exactly-once computation that sends a non-transactional external email can still produce duplicate emails during recovery.

Idempotency means that repeating the same operation has the same effect as performing it once. A payment update based on a unique transaction identifier can be made idempotent by rejecting a previously applied identifier. Deduplication requires storing identifiers for an appropriate period and managing the risk of collisions or incorrect reuse.

Correctness is broader than delivery semantics. A system may deliver every event exactly once and still produce a wrong result if event timestamps are unreliable, schemas change unexpectedly, joins use stale reference data, or late events are ignored. Architecture documentation should therefore state assumptions about ordering, duplicates, time, state, and corrections.

Complex Event Processing

Complex event processing (CEP) identifies meaningful situations from combinations, sequences, absences, and relationships among events. It is different from simply calculating an aggregate. A rule such as “three failed logins followed by a successful login from a new location within ten minutes” describes a temporal pattern.

CEP commonly uses:

  • sequence detection, where events must occur in a specified order;
  • repetition, where several similar events occur within a period;
  • correlation, where events share an identifier such as account or device;
  • absence detection, where an expected event does not occur;
  • threshold rules, where a measurement crosses a limit; and
  • temporal joins, where events from different streams are related by time and key.

A CEP rule must specify event types, correlation keys, time constraints, acceptable gaps, competing matches, and the action produced by a match. Poorly designed rules can generate alert fatigue. A rule that is technically sensitive but operationally unactionable may reduce trust in the entire analytical system.

CEP can be combined with machine-learning models. A model may estimate the probability of fraud, while a CEP layer combines that score with transaction velocity, device changes, and prior outcomes. The model and the rule engine should be monitored separately because a low-quality model cannot be repaired by adding more event patterns.

Designing a Streaming Solution

A disciplined design process begins with the decision or action that the stream must support. The architect should identify the event source, required latency, expected volume, acceptable delay, correctness requirements, retention period, and destination.

The following design questions are particularly important:

1. What constitutes an event, and who owns its definition?

2. Which fields are required for correlation, ordering, deduplication, and audit?

3. Is event time or processing time the basis of the calculation?

4. What level of out-of-order and late data is expected?

5. Which keys determine partitioning and state ownership?

6. What delivery guarantee is justified by the business risk?

7. What happens when the destination is unavailable?

8. Can results be replayed, corrected, and audited?

9. How will schema changes be governed?

10. Which metrics indicate that the system is healthy and useful?

Operational metrics should include throughput, end-to-end latency, consumer lag, processing failures, checkpoint duration, state size, watermark progress, late-event rate, duplicate rate, and output error rate. Business metrics may include prevented losses, response time, alert precision, conversion improvement, or service-level compliance. Monitoring only infrastructure is insufficient; a technically healthy stream can still produce poor decisions.

Practical and Industry Applications

In financial services, event streams support fraud detection, transaction monitoring, market surveillance, and payment authorization. The high cost of false negatives must be balanced against customer friction caused by false positives.

In manufacturing, sensors can produce temperature, vibration, pressure, and energy events. Windowed statistics and anomaly detection can identify maintenance needs. Safety applications require conservative failure handling, independent controls, and clear escalation procedures.

In telecommunications and cloud operations, logs, traces, and metrics are analysed continuously to identify service degradation. Correlating events across services can distinguish a single local failure from a wider incident.

In retail and digital platforms, clickstreams, cart updates, inventory changes, and transactions support recommendations, dynamic offers, stock alerts, and customer-journey analysis. Event-time processing is important when users interact through unreliable mobile connections.

In transportation, location and status events can support route optimization, estimated arrival times, fleet maintenance, and safety alerts. Location data also creates significant privacy responsibilities.

Illustrative Case Study: Real-Time Payment Risk Detection

Background

Consider a fictional digital bank, Meridian Bank, processing card and account-to-account payments through mobile and online channels. Its existing fraud process runs in hourly batches. The process identifies suspicious activity, but investigators may receive alerts after a fraudulent transfer has been completed.

Situation and Problem

The bank wants to evaluate transactions within a few seconds. A single transaction may not appear suspicious, but a sequence of events may be concerning: multiple payments from a newly registered device, rapid transfers to new beneficiaries, or an unusual geographic change. The bank must also avoid blocking legitimate customers during travel or periods of high activity.

Analysis

The bank identifies transaction authorization, login, device registration, beneficiary creation, and customer-profile events as important streams. Each event receives a unique identifier and an event timestamp. Events are partitioned by customer or account where ordering is required, while high-volume operational streams use keys designed to distribute load.

The risk decision requires both recent aggregates and temporal patterns. A sliding ten-minute window calculates transfer count and total value. A session or sequence rule identifies a new beneficiary followed by several high-value transfers. A reference-data join adds account status and known device information. Because mobile events may arrive late, the design uses event time, watermarks, and an allowed lateness period.

Approach and Solution

A message broker receives the events and retains them long enough to support replay and investigation. Stream processors validate schemas, remove duplicates using event identifiers, enrich events, and maintain keyed state. A risk service combines rules with a statistical risk score. Transactions are assigned one of three outcomes: approve, challenge with additional authentication, or hold for review.

At-least-once delivery is selected for the analytical pipeline because losing a transaction is unacceptable. Idempotent updates and transaction identifiers prevent duplicate risk effects. The final authorization decision uses a carefully controlled interface so that retries do not create multiple holds or customer notifications. Checkpoints allow processors to recover state, and a dead-letter stream captures invalid events for investigation.

The bank monitors processing latency, consumer lag, late-event rates, model performance, challenge rates, confirmed fraud, and customer complaints. Rules are reviewed when alert volume rises or fraud tactics change.

Outcome

The system moves detection closer to the time of payment and provides investigators with a more complete event history. Its value does not depend solely on lower latency: replayable events improve auditability, while separate rule and model monitoring supports controlled change. The bank continues to use batch processing for reconciliation and historical model training because streaming does not eliminate the need for offline analytics.

Learning Points

The case demonstrates that real-time analytics is an end-to-end design problem. Event definitions, timestamps, partitioning, state, delivery semantics, human workflows, and governance must be designed together. It also shows that exactly-once language should be applied to a defined processing boundary rather than treated as a guarantee for every external action.

Challenges, Risks, and Common Mistakes

A frequent mistake is equating low processing latency with useful real-time analytics. A result produced quickly from incomplete or incorrectly timed data may be worse than a slower, accurate result. Another mistake is using processing time for a business question defined in event time.

Unbounded state is a major technical risk. Retaining every key indefinitely can exhaust resources, especially when identifiers are high-cardinality or maliciously generated. State expiration and capacity limits should be explicit.

Poor partition-key selection can create hotspots or break required ordering. Excessive partitioning increases coordination and operational cost, while insufficient partitioning limits throughput.

Schema evolution is another source of failure. Producers and consumers should agree on compatibility rules, versioning, required fields, and deprecation practices. A malformed event should not silently corrupt downstream results.

Streaming systems can also amplify errors. An incorrect rule may generate thousands of alerts; a faulty enrichment service may contaminate every output; and an incorrectly configured retry may duplicate external actions. Testing should include out-of-order events, duplicates, missing data, clock errors, processor failure, broker unavailability, and recovery from checkpoints.

Governance, Ethics, and Professional Responsibility

Real-time analytics frequently processes personal, financial, location, or behavioural data. Organizations should collect only information necessary for a defined purpose, control access, encrypt data in transit and at rest, define retention periods, and maintain audit trails. Event replay must not become an excuse for indefinite retention.

Automated decisions can create discrimination or unjustified exclusion. A fraud system may disproportionately challenge particular customer groups if its data or rules reflect historical bias. Organizations should monitor outcomes across relevant populations, provide escalation and review mechanisms, and document the rationale for high-impact decisions.

Data provenance is important. Analysts and investigators should be able to determine which events, versions of rules, reference data, and model outputs contributed to a decision. Where regulations require explanation, retention, correction, or human review, the streaming architecture must support those obligations.

Professional practice also requires clear ownership. Data producers own event quality; platform teams own reliability; analytics teams own transformations and models; and business or risk functions own decision policies. Shared responsibility should be documented rather than assumed.

Technology and Emerging Developments

Modern streaming platforms increasingly combine durable event logs, stateful processing, SQL-like interfaces, lakehouse storage, and managed cloud services. A common pattern is to use the stream for immediate decisions while writing the same events to durable analytical storage for historical analysis and replay.

Stream processing is also becoming more closely integrated with machine learning. Online feature computation can maintain recent behavioural signals, while model-serving systems score events as they arrive. Feature definitions must remain consistent between training and serving; otherwise, data leakage or training-serving skew can occur.

Other important developments include incremental materialized views, adaptive resource allocation, event-driven data products, and observability focused on data quality rather than infrastructure alone. Large language models may assist operators by summarizing incidents or querying event histories, but their outputs should not bypass deterministic controls in high-risk decisions.

The boundary between batch and streaming is becoming less rigid. Many platforms support unified processing models in which historical data is replayed through the same transformations used for live data. This can simplify testing and recovery, but it does not remove the need to understand time, state, duplicates, and external side effects.

Best Practices for Professional Implementation

A sound implementation generally follows these principles:

  • Define events with stable identifiers, explicit timestamps, ownership, and versioned schemas.
  • Select event-time or processing-time semantics according to the decision, not merely implementation convenience.
  • Choose partition keys that satisfy ordering needs without creating hotspots.
  • Design state retention, checkpointing, recovery, and replay before production deployment.
  • Use idempotency and deduplication for at-least-once workflows.
  • Treat late data, invalid data, and unavailable dependencies as normal operating conditions.
  • Separate provisional results from final or corrected results.
  • Test failure, delay, duplication, reordering, and schema evolution explicitly.
  • Monitor both technical health and business effectiveness.
  • Apply privacy, security, fairness, auditability, and access controls throughout the event lifecycle.
  • Keep automated actions proportional to risk and provide human intervention where consequences are significant.

Chapter Summary

Streaming and real-time analytics process continuously arriving events to support timely decisions and actions. Event-driven architecture separates producers from consumers through brokers and enables multiple applications to use shared event streams. Stream processors transform, enrich, aggregate, join, and analyse unbounded data.

Correct temporal reasoning is essential. Event time describes when an event occurred, whereas processing time describes when it was handled. Watermarks and allowed lateness help systems produce event-time results despite out-of-order arrivals. Tumbling, sliding, hopping, session, and count-based windows provide different ways to analyse finite portions of an unbounded stream.

State management, checkpoints, partitioning, and replay determine whether a system can scale and recover correctly. At-most-once, at-least-once, and exactly-once semantics represent different trade-offs, and idempotency is central to reliable side effects. Complex event processing identifies patterns, sequences, thresholds, and absences across correlated events.

Effective streaming systems combine technical architecture with operational monitoring, governance, privacy, fairness, and human decision processes. The best design is not necessarily the one with the lowest latency; it is the one that delivers sufficiently timely, accurate, explainable, secure, and actionable results for its context.

Key Terms

Event: A recorded occurrence that represents a change, action, observation, or state transition.

Event-driven architecture: An architecture in which components communicate through published events rather than direct, tightly coupled calls.

Message broker: Infrastructure that receives, stores, and distributes messages or events to consumers.

Topic: A logical named stream of related events in a broker.

Partition: An ordered subdivision of a topic used for parallelism and scalability.

Consumer group: A set of consumers that collectively process partitions of a stream.

Stream processor: A system that continuously transforms and analyses incoming events.

Event time: The time at which an event actually occurred.

Processing time: The time at which a processor handles an event.

Watermark: An estimate that events before a specified event-time boundary are unlikely to arrive.

Window: A finite logical subset of an unbounded stream used for computation.

Tumbling window: A fixed, non-overlapping window.

Sliding window: A fixed-length window that advances at regular intervals and may overlap.

Session window: A window defined by periods of activity separated by inactivity gaps.

State: Information retained by a processor across multiple events.

Checkpoint: A durable record of processing progress and, often, operator state.

Idempotency: The property that repeated execution of an operation produces the same logical effect as one execution.

At-least-once delivery: A guarantee that events are not intentionally lost, although duplicates may occur.

Complex event processing: Detection of meaningful patterns or relationships across multiple events over time.

Consumer lag: The difference between the latest available event position and the position processed by a consumer.

Review Questions

Short Answer Questions

1. What is the purpose of a message broker in an event-driven architecture?

2. Distinguish event time from processing time.

3. What problem does a watermark address?

4. Define idempotency in the context of stream processing.

5. What is a session window, and when might it be useful?

Long Answer Questions

1. Explain the architecture of a streaming analytics system from event production to final action.

2. Compare tumbling, sliding, hopping, session, and count-based windows with suitable applications.

3. Discuss state management, checkpointing, replay, and recovery in a distributed stream processor.

4. Compare at-most-once, at-least-once, and exactly-once delivery semantics and explain their practical trade-offs.

5. Explain how complex event processing differs from ordinary aggregation and describe its major pattern types.

Analytical / Application Questions

1. A logistics company receives vehicle-location events from mobile devices that may reconnect after several minutes. Should its estimated-arrival calculation use event time or processing time? Explain the required handling of watermarks and late events.

2. Design a streaming solution for detecting repeated failed logins followed by a successful login from a new device. Specify the event streams, correlation key, time constraints, state, output action, and delivery considerations.

3. A real-time dashboard shows accurate results during normal operation but produces duplicates after processor restart. Diagnose likely causes and propose changes involving checkpoints, delivery semantics, idempotency, and sink design.

22. Multiple Choice Questions with Answers

Question 1

A retailer wants to calculate the number of purchases that actually occurred between 10:00 and 10:05, including purchases uploaded late by mobile devices. Which time concept should primarily define the windows?

A. Processing time

B. Event time

C. Checkpoint time

D. Consumer acknowledgement time

Correct Answer: B. Event time

Question 2

What is the main purpose of a watermark in event-time stream processing?

A. To encrypt events before storage

B. To identify the broker partition with the highest load

C. To estimate when the system may finalize results for earlier event times

D. To guarantee that no event will ever arrive late

Correct Answer: C. To estimate when the system may finalize results for earlier event times

Question 3

A stream processor must preserve the order of transactions for each customer while processing many customers in parallel. Which design choice is most appropriate?

A. Partition events using the customer identifier

B. Place all events in one partition

C. Use processing time as the partition key

D. Randomly assign every event to a new partition

Correct Answer: A. Partition events using the customer identifier

Question 4

Which statement best describes at-least-once processing?

A. Every event is processed exactly once with no possibility of duplication

B. Events may be lost, but none can be duplicated

C. Events are not intentionally lost, but retries may cause duplicate processing

D. Events are processed only after a human approves them

Correct Answer: C. Events are not intentionally lost, but retries may cause duplicate processing

Question 5

Which window is most suitable for analysing each user interaction period separated by 30 minutes of inactivity?

A. Tumbling window

B. Session window

C. Count-based window

D. Fixed global window

Correct Answer: B. Session window

Question 6

A payment service retries an update after a timeout. Which property prevents the retry from applying the payment twice?

A. Partitioning

B. Idempotency

C. Watermarking

D. Sliding-window evaluation

Correct Answer: B. Idempotency

Question 7

Which situation is an example of complex event processing rather than a simple aggregation?

A. Calculating total sales every five minutes

B. Computing the average temperature in a window

C. Detecting three failed logins followed by a successful login from a new device

D. Counting messages received by a topic

Correct Answer: C. Detecting three failed logins followed by a successful login from a new device

Question 8

A stream application retains state for every identifier ever observed and eventually exhausts memory. Which control is most directly needed?

A. State expiration or bounded retention

B. Increasing the number of event types

C. Removing all checkpoints

D. Changing every window to a global window

Correct Answer: A. State expiration or bounded retention

Question 9

Why can exactly-once processing fail to prevent duplicate external notifications?

A. Exactly-once applies only to a defined processing boundary and may not cover a non-transactional notification service

B. Exactly-once always discards late events

C. Exactly-once cannot use message brokers

D. Exactly-once requires processing time rather than event time

Correct Answer: A. Exactly-once applies only to a defined processing boundary and may not cover a non-transactional notification service

Question 10

Which metric most directly indicates that a consumer is falling behind the incoming stream?

A. Schema version

B. Consumer lag

C. Window size

D. Event identifier length

Correct Answer: B. Consumer lag

Big Data and Cloud Analytics
Chapter 8 of 15