Dart Streams in Depth: Broadcast Streams, Transformers, and Backpressure Handling
Asynchronous programming in Dart revolves around two fundamental primitives: Future (representing a single computation or value delivered asynchronously in the future) and Stream (representing a sequence of asynchronous data events over time).
While Future is straightforward for one-shot operations like HTTP requests, real-world interactive systemsβsuch as real-time WebSocket communication, live sensor feeds, GPS tracking, database change-listeners, reactive state management (BLoC/Cubit), and user input processingβrequire continuous, event-driven data flow. That is the domain of Dart Streams.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β The Dart Async Grid β
βββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββ€
β β Single Value β Sequence / Stream β
βββββββββββββββββββββββββΌββββββββββββββββββββββββββββββΌββββββββββββββββββββββββ€
β Synchronous (Pull) β T / Object β Iterable<T> β
β Asynchronous (Push) β Future<T> β Stream<T> β
βββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββ
However, streams are also one of the most common sources of subtle bugs in Dart and Flutter applications: unhandled backpressure leading to unbounded memory growth, Bad state: Stream has already been listened to crashes, lost broadcast events, and memory leaks from uncancelled subscriptions.
In this deep dive, we will explore Dart Streams from the underlying runtime mechanics to production-grade architectures:
- Foundations of Reactive Data in Dart: Push vs. pull, the three event types, and the subscription lifecycle.
- Single-Subscription vs. Broadcast Streams: Architectural differences, buffering, multi-subscriber topologies, and lifecycle semantics.
StreamControllerUnder the Hood: Cold on-demand generation, hot feeds, synchronous controllers, and event loop microtask semantics.- Custom
StreamTransformerArchitecture: Constructing stateless handlers and statefulStreamTransformerBasepipelines. - Backpressure & Flow Control: Solving fast producer vs. slow consumer bottlenecks with custom debouncing, throttling, and chunk buffering.
- Advanced Stream Combinators:
asyncMap,switchMap,exhaustMap,concatMap, and multi-stream synchronization. - Subscription Hygiene & Memory Leak Prevention: Disposal rules, composite subscriptions, and Flutter
StreamBuilderanti-patterns. - Production Decision Matrix & Best Practices: Architectural decision tree and operational guidelines.
1. Foundations of Reactive Data in Dartβ
Push vs. Pull: Why Streams are Asynchronous Sequencesβ
An Iterable<T> represents a pull-based sequence. The consumer controls execution: nothing happens until the consumer calls .iterator.moveNext() or iterates in a for-in loop. Execution is synchronous and blocking on the current thread.
A Stream<T> represents a push-based sequence. The producer (such as a WebSocket, timer, or UI gesture engine) produces events whenever they occur and pushes them to registered listeners asynchronously via the Dart event queue:
// Synchronous Pull (Iterable)
Iterable<int> syncRange(int count) sync* {
for (int i = 0; i < count; i++) {
yield i; // Consumer pulls value when ready
}
}
// Asynchronous Push (Stream)
Stream<int> asyncRange(int count) async* {
for (int i = 0; i < count; i++) {
await Future.delayed(const Duration(milliseconds: 100));
yield i; // Producer pushes value to listener when ready
}
}
Pull (Iterable): Consumer βββββββΊ moveNext() βββββββΊ Producer generates item
Push (Stream): Producer βββββββΊ push(event) ββββββΊ Consumer onData(item)
The Three Stream Eventsβ
Every stream in Dart transmits exactly three types of notifications:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Stream Event Sequence β
β β
β ββββΊ [ Data Event: 1 ] βββΊ [ Data Event: 2 ] βββΊ [ Error Event ] βββΊ [ Done ]
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
- Data Event (
onData): A value of typeTsuccessfully emitted. A stream can emit zero, one, or infinitely many data events. - Error Event (
onError): An exception or error object emitted alongside an optionalStackTrace. An error event does NOT automatically terminate the stream; subsequent data or error events can still follow unlesscancelOnError: trueis configured. - Done Event (
onDone): A sentinel event signaling that the stream has finished producing events. No further events will ever be emitted.
final Stream<int> stream = Stream.fromIterable([1, 2, 3]);
final StreamSubscription<int> subscription = stream.listen(
(data) {
print('Received data: $data');
},
onError: (error, stackTrace) {
print('Received error: $error');
},
onDone: () {
print('Stream completed successfully.');
},
cancelOnError: false, // Default is false: continues listening after errors
);
The Anatomy of StreamSubscriptionβ
When you call stream.listen(), Dart creates a StreamSubscription<T>. The subscription is the active link between the stream producer and your listener callback. It provides imperative controls over data flow:
// Pausing and resuming data flow
subscription.pause(); // Tells producer to pause or buffer incoming events
print('Is paused: ${subscription.isPaused}');
// Resume emission
subscription.resume();
// Cancelling the subscription
await subscription.cancel(); // Detaches listener and frees resources
[!IMPORTANT] If you pause a single-subscription stream, the underlying source (like a file stream or socket) will stop reading from the OS kernel buffer. However, if you pause a broadcast stream, events generated during the paused state might be discarded or buffered depending on controller implementation!
2. Single-Subscription vs. Broadcast Streamsβ
Dart features two fundamentally distinct categories of streams. Choosing the wrong type is one of the most common causes of runtime crashes in Flutter apps.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Single-Subscription Stream β
β β
β [ Producer ] βββββββββββββββ Buffer / Flow ββββββββββββββββΊ [ Listener 1 ] β
β (One listener only. Buffers events before listen. Cannot re-listen.) β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Broadcast Stream β
β βββΊ [ Listener 1 ]
β βββΊ [ Listener 2 ]
β [ Producer ] βββββββββββββββ Push Directly ββββββββββββββββββΌββΊ [ Listener 3 ]
β (Multiple listeners. No buffer. Missed events are lost.) βββΊ [ Listener N ]
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Single-Subscription Streams (Cold Streams)β
A single-subscription stream is designed for delivering a discrete sequence of data in order (e.g., reading a file, downloading an HTTP body, querying a database cursor).
Core Characteristics:β
- Strictly Single Listener: Only one subscriber can listen at any point in its lifetime.
- Pre-Listen Buffering: Events emitted before a listener attaches are safely held in an internal buffer and delivered as soon as
.listen()is invoked. - No Re-Listening: Even after the first listener cancels or completes, calling
.listen()a second time throws a runtime error:StateError (Bad state: Stream has already been listened to.)
import 'dart:async';
void demonstrateSingleSubscription() async {
final controller = StreamController<String>();
// Buffer events before anyone is listening
controller.add('Event 1');
controller.add('Event 2');
// First listener attaches and receives buffered events
final sub1 = controller.stream.listen((val) => print('Sub 1 received: $val'));
await Future.delayed(Duration.zero); // Drain microtasks
await sub1.cancel();
// β CRASH: Throws StateError! Cannot re-listen to a single-subscription stream.
try {
controller.stream.listen((val) => print('Sub 2 received: $val'));
} catch (e) {
print('Caught expected error: $e');
}
await controller.close();
}
Broadcast Streams (Hot Streams)β
A broadcast stream is designed for independent, concurrent consumers observing fire-and-forget events (e.g., UI touch coordinates, mouse movements, device sensor updates, state change notifications).
Core Characteristics:β
- Multiple Concurrent Listeners: Any number of listeners can subscribe and unsubscribe independently at any time.
- Zero Pre-Listen Buffering: Events pushed to a broadcast stream when there are zero active listeners are dropped immediately.
- No Playback: New subscribers only receive events emitted after they attach. They never receive past events.
void demonstrateBroadcast() async {
final controller = StreamController<String>.broadcast();
// Event added with NO active listeners -> DROPPED immediately!
controller.add('Dropped Event');
// First listener subscribes
final sub1 = controller.stream.listen((val) => print('Sub 1: $val'));
controller.add('Broadcast Event A');
// Second listener subscribes later
final sub2 = controller.stream.listen((val) => print('Sub 2: $val'));
controller.add('Broadcast Event B');
// Clean up
await sub1.cancel();
await sub2.cancel();
await controller.close();
}
// Output:
// Sub 1: Broadcast Event A
// Sub 1: Broadcast Event B
// Sub 2: Broadcast Event B
Transforming Single-Subscription into Broadcast: asBroadcastStream()β
You can convert any single-subscription stream into a broadcast stream using the .asBroadcastStream() method:
Stream<List<int>> fileByteStream = file.openRead();
Stream<List<int>> broadcastBytes = fileByteStream.asBroadcastStream(
onListen: (subscription) => print('First listener attached to file broadcast'),
onCancel: (subscription) => print('All listeners detached from file broadcast'),
);
[!WARNING] When using
stream.asBroadcastStream(), the underlying single-subscription stream starts listening to its source as soon as the first broadcast listener arrives. If that first listener cancels, the underlying subscription stays alive unless you explicitly handleonCancelor all listeners detach!
Comprehensive Comparison Matrixβ
| Feature | Single-Subscription Stream | Broadcast Stream |
|---|---|---|
| Max Subscribers | Exactly 1 over its entire lifetime | Unlimited (0 to $\infty$) |
| Event Delivery | Guaranteed; buffered before listening | Real-time; dropped if no active listeners |
| Re-listening | β Throws StateError | β Allowed at any time |
| Typical Analogy | Reading a book / Audio file | Live FM Radio broadcast |
| Primary Use Cases | File I/O, HTTP requests, DB queries | UI Gestures, WebSockets, Sensor feeds |
| Pause Behavior | Source pauses event generation | Listener ignores incoming events |
3. StreamController Under the Hoodβ
StreamController<T> is the primary tool for creating and managing custom streams. It provides an EventSink<T> for pushing data and exposes a Stream<T> for consumers.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β StreamController<T> β
β β
β βββββββββββββββββββββββββββββββββββββ β
β controller.sink β .add(T) β β
β (EventSink<T>) β .addError(Object, StackTrace?) β β
β β .close() β β
β βββββββββββββββββββ¬ββββββββββββββββββ β
β β β
β βΌ β
β βββββββββββββββββββββββββββββββββββββ β
β controller.stream β .listen(...) β β
β (Stream<T>) β .map(), .where(), .transform() β β
β βββββββββββββββββββββββββββββββββββββ β
β β
β Lifecycle Hooks: onListen, onPause, onResume, onCancel β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Implementing Cold, On-Demand Producers via Lifecycle Callbacksβ
The most resource-efficient streams only consume CPU, battery, or network bandwidth when there is an active subscriber. StreamController provides four lifecycle hooks to manage resources:
onListen: Fired when the subscriber count transitions from0 -> 1. Start timers, open sockets, or initialize hardware.onPause: Fired when the active subscriber pauses data delivery. Pause internal timers or sensors.onResume: Fired when the active subscriber resumes data delivery.onCancel: Fired when the subscriber count drops to0. Stop timers, close sockets, and release resources.
Here is an enterprise-grade, memory-efficient sensor stream generator:
import 'dart:async';
class HardwareSensorStream {
Timer? _pollingTimer;
int _counter = 0;
late final StreamController<double> _controller;
HardwareSensorStream() {
_controller = StreamController<double>(
onListen: _startHardwarePolling,
onPause: _pauseHardwarePolling,
onResume: _startHardwarePolling,
onCancel: _stopHardwarePolling,
);
}
Stream<double> get stream => _controller.stream;
void _startHardwarePolling() {
print('[Hardware] Starting sensor polling timer...');
_pollingTimer = Timer.periodic(const Duration(milliseconds: 50), (timer) {
final simulatedVoltage = 3.3 + (_counter++ % 10) * 0.05;
if (!_controller.isClosed) {
_controller.add(simulatedVoltage);
}
});
}
void _pauseHardwarePolling() {
print('[Hardware] Pausing sensor polling...');
_pollingTimer?.cancel();
}
void _stopHardwarePolling() {
print('[Hardware] Stopping sensor polling & freeing hardware resources...');
_pollingTimer?.cancel();
_pollingTimer = null;
_counter = 0;
}
Future<void> dispose() async {
await _controller.close();
}
}
Synchronous Controllers (sync: true) vs. Asynchronous Controllersβ
By default, StreamController() is asynchronous. When you call controller.add(event), the event is scheduled as a microtask and delivered on the next turn of the event loop.
If you pass StreamController(sync: true), calling controller.add(event) executes the listener's onData callback synchronously on the current execution stack, immediately interrupting the current function:
void demonstrateSyncVsAsyncController() {
final asyncCtrl = StreamController<String>();
final syncCtrl = StreamController<String>(sync: true);
asyncCtrl.stream.listen((v) => print('Async Received: $v'));
syncCtrl.stream.listen((v) => print('Sync Received: $v'));
print('1. Before Adding');
asyncCtrl.add('A');
syncCtrl.add('B'); // Hits the listener IMMEDIATELY!
print('2. After Adding');
asyncCtrl.close();
syncCtrl.close();
}
// Console Output:
// 1. Before Adding
// Sync Received: B <-- Executed synchronously inside syncCtrl.add('B')!
// 2. After Adding
// Async Received: A <-- Executed later in microtask queue
[!CAUTION] Avoid
sync: truein general application code! Synchronous controllers break Dart's normal asynchronous invariants. If a synchronous controller emits an event inside an event handler, it can cause re-entrancy bugs, unexpected stack overflow crashes, and violate the principle of least astonishment. Usesync: trueonly when building low-level performance-critical bridge primitives orStreamTransformersinks.
4. Custom StreamTransformer: The Functional Pipelineβ
StreamTransformer<S, T> is the standard abstraction for transforming an input stream of type S into an output stream of type T. It powers built-in operators like .map(), .where(), .cast(), and encoding pipelines like utf8.decoder.
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Stream Transformation Pipeline β
β β
β Stream<List<int>> (Raw Bytes) β
β β β
β βΌ .transform(utf8.decoder) β
β Stream<String> (Decoded Unicode Characters) β
β β β
β βΌ .transform(const LineSplitter()) β
β Stream<String> (Individual Lines) β
β β β
β βΌ .transform(CustomJsonPacketTransformer()) β
β Stream<Packet> (Parsed Business Objects) β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Approach 1: Lightweight Transformers with StreamTransformer.fromHandlers()β
For simple filtering, mapping, and error recovery, use StreamTransformer.fromHandlers():
import 'dart:async';
/// Transformer that validates integers and discards negative values while
/// converting valid integers into formatted strings.
final sanitizeAndFormatTransformer = StreamTransformer<int, String>.fromHandlers(
handleData: (int data, EventSink<String> sink) {
if (data < 0) {
// Discard invalid data or convert into error
sink.addError(ArgumentError('Negative values are not permitted: $data'));
} else {
sink.add('Value: 0x${data.toRadixString(16).padLeft(4, '0')}');
}
},
handleError: (Object error, StackTrace stackTrace, EventSink<String> sink) {
print('Intercepted error in pipeline: $error');
// We can recover by emitting a fallback value or forward the error:
sink.add('Fallback: 0x0000');
},
handleDone: (EventSink<String> sink) {
print('Pipeline complete. Closing sink.');
sink.close();
},
);
void main() {
final source = Stream<int>.fromIterable([10, -5, 255, 4096]);
source
.transform(sanitizeAndFormatTransformer)
.listen(
(data) => print('Data: $data'),
onError: (err) => print('Error: $err'),
onDone: () => print('Done!'),
);
}
Approach 2: Production Stateful Transformers with StreamTransformerBaseβ
When transformation logic requires retaining state across chunks (e.g., parsing binary frames with length headers, assembling fragmented packets, or rolling window calculations), you must implement StreamTransformerBase<S, T>.
Let's build a real-world Packet Framing Transformer that decodes a stream of arbitrary byte chunks where messages are separated by a delimiter byte (0x00):
import 'dart:async';
import 'dart:convert';
import 'dart:typed_data';
/// Splits incoming arbitrary byte chunks by a 0x00 delimiter byte
/// and emits complete UTF-8 strings.
class DelimitedMessageDecoder extends StreamTransformerBase<List<int>, String> {
final int delimiter;
const DelimitedMessageDecoder({this.delimiter = 0x00});
Stream<String> bind(Stream<List<int>> stream) {
return Stream<String>.eventTransformed(
stream,
(EventSink<String> outputSink) => _DelimitedMessageSink(outputSink, delimiter),
);
}
}
class _DelimitedMessageSink implements EventSink<List<int>> {
final EventSink<String> _outputSink;
final int _delimiter;
final List<int> _accumulator = [];
_DelimitedMessageSink(this._outputSink, this._delimiter);
void add(List<int> chunk) {
for (int i = 0; i < chunk.length; i++) {
final byte = chunk[i];
if (byte == _delimiter) {
// Delimiter encountered: flush accumulated buffer as complete string
if (_accumulator.isNotEmpty) {
try {
final message = utf8.decode(_accumulator);
_outputSink.add(message);
} catch (e, st) {
_outputSink.addError(FormatException('Failed to decode UTF8 frame: $e'), st);
}
_accumulator.clear();
}
} else {
_accumulator.add(byte);
}
}
}
void addError(Object error, [StackTrace? stackTrace]) {
_outputSink.addError(error, stackTrace);
}
void close() {
// Flush remaining buffer if not empty on stream completion
if (_accumulator.isNotEmpty) {
try {
_outputSink.add(utf8.decode(_accumulator));
} catch (e, st) {
_outputSink.addError(FormatException('Malformed trailing bytes: $e'), st);
}
_accumulator.clear();
}
_outputSink.close();
}
}
Running the Chunk Decoder:β
void main() async {
// Simulating fragmented TCP/WebSocket packets
final rawChunks = [
[72, 101, 108, 108, 111], // "Hello" (no delimiter)
[32, 87, 111, 114, 108, 100, 0, 83, 101], // " World\0Se" (delimiter in middle)
[99, 111, 110, 100, 0], // "cond\0"
];
final stream = Stream<List<int>>.fromIterable(rawChunks);
await for (final message in stream.transform(const DelimitedMessageDecoder())) {
print('Received Decoded Message: "$message"');
}
}
// Output:
// Received Decoded Message: "Hello World"
// Received Decoded Message: "Second"
5. Backpressure, Rate Limiting & Flow Controlβ
What is Backpressure?β
In an event-driven system, backpressure occurs when the producer emits events faster than the consumer can process them.
Fast Producer (Camera / Socket / Keystrokes)
βββββΊ [Event] βββΊ [Event] βββΊ [Event] βββΊ [Event] βββΊ [Event] βββΊ (Queue: 100,000 items!)
β
Slow Consumer (Disk write / Heavy JSON / Network POST) βΌ
ββββββββββββββββββββΊ [ Processing Event (200ms) ] βββββββββββΊ Out Of Memory (OOM) Crash!
If left unmanaged, the unbounded event queue consumes available heap memory until the Dart VM or OS terminates the process.
To manage backpressure, we employ three rate-limiting patterns:
- Debounce: Wait for a quiet period before emitting the latest value.
- Throttle: Emit the first value immediately, then ignore values for a duration window.
- Buffer / Batch: Accumulate values into fixed-size lists or time buckets.
Source Events: ββAββBβCββββββDββββββββEββFββGββββΊ
Debounce (300ms): βββββββCβββββββββDββββββββββββGβββΊ (Emits after silence)
Throttle (300ms): ββAβββββββββββDββββββββEββββββββββΊ (Emits first, silences interval)
Buffer (Count = 3): ββββββββββ[A,B,C]βββββββββ[D,E,F]βΊ (Emits grouped arrays)
Pattern 1: High-Performance Debounce Transformerβ
A debounce operator postpones emitting an event until a specified Duration has elapsed with no other events arriving. It is ideal for search inputs, auto-save triggers, and window resizing.
// debounce_transformer.dart
import 'dart:async';
class DebounceTransformer<T> extends StreamTransformerBase<T, T> {
final Duration duration;
const DebounceTransformer(this.duration);
Stream<T> bind(Stream<T> stream) {
return Stream<T>.eventTransformed(
stream,
(EventSink<T> sink) => _DebounceSink<T>(sink, duration),
);
}
}
class _DebounceSink<T> implements EventSink<T> {
final EventSink<T> _outputSink;
final Duration _duration;
Timer? _timer;
T? _lastValue;
bool _hasValue = false;
bool _isSourceClosed = false;
_DebounceSink(this._outputSink, this._duration);
void add(T data) {
_lastValue = data;
_hasValue = true;
_timer?.cancel();
_timer = Timer(_duration, () {
if (_hasValue) {
_outputSink.add(_lastValue as T);
_hasValue = false;
_lastValue = null;
}
if (_isSourceClosed) {
_outputSink.close();
}
});
}
void addError(Object error, [StackTrace? stackTrace]) {
_outputSink.addError(error, stackTrace);
}
void close() {
_isSourceClosed = true;
if (_timer == null || !_timer!.isActive) {
if (_hasValue) {
_outputSink.add(_lastValue as T);
}
_outputSink.close();
}
}
}
/// Convenience extension method
extension StreamDebounceExtension<T> on Stream<T> {
Stream<T> debounce(Duration duration) => transform(DebounceTransformer<T>(duration));
}
Pattern 2: Throttle / Rate-Limiter Transformerβ
A throttle operator emits the initial event immediately, then discards all subsequent events for the duration of the cooldown window. It is ideal for button click debouncing (preventing double submissions) and high-frequency GPS or scroll updates.
// throttle_transformer.dart
import 'dart:async';
class ThrottleTransformer<T> extends StreamTransformerBase<T, T> {
final Duration duration;
final bool trailing;
const ThrottleTransformer(this.duration, {this.trailing = false});
Stream<T> bind(Stream<T> stream) {
return Stream<T>.eventTransformed(
stream,
(EventSink<T> sink) => _ThrottleSink<T>(sink, duration, trailing),
);
}
}
class _ThrottleSink<T> implements EventSink<T> {
final EventSink<T> _outputSink;
final Duration _duration;
final bool _trailing;
Timer? _cooldownTimer;
T? _trailingValue;
bool _hasTrailingValue = false;
_ThrottleSink(this._outputSink, this._duration, this._trailing);
void add(T data) {
if (_cooldownTimer == null || !_cooldownTimer!.isActive) {
// Leading edge: emit immediately
_outputSink.add(data);
_startCooldown();
} else if (_trailing) {
// Store latest value to emit at the end of the window
_trailingValue = data;
_hasTrailingValue = true;
}
}
void _startCooldown() {
_cooldownTimer = Timer(_duration, () {
if (_trailing && _hasTrailingValue) {
_outputSink.add(_trailingValue as T);
_trailingValue = null;
_hasTrailingValue = false;
_startCooldown();
}
});
}
void addError(Object error, [StackTrace? stackTrace]) {
_outputSink.addError(error, stackTrace);
}
void close() {
_cooldownTimer?.cancel();
_outputSink.close();
}
}
extension StreamThrottleExtension<T> on Stream<T> {
Stream<T> throttle(Duration duration, {bool trailing = false}) =>
transform(ThrottleTransformer<T>(duration, trailing: trailing));
}
Pattern 3: Chunk / Buffer Transformer (Batching)β
Batching groups individual events into chunks of size $N$ or flushes them periodically. It is essential for database bulk inserts, batch analytics logging, and telemetry aggregators:
// buffer_transformer.dart
import 'dart:async';
class BufferCountTransformer<T> extends StreamTransformerBase<T, List<T>> {
final int count;
const BufferCountTransformer(this.count) : assert(count > 0);
Stream<List<T>> bind(Stream<T> stream) {
return Stream<List<T>>.eventTransformed(
stream,
(EventSink<List<T>> sink) => _BufferCountSink<T>(sink, count),
);
}
}
class _BufferCountSink<T> implements EventSink<T> {
final EventSink<List<T>> _outputSink;
final int _count;
List<T> _buffer = [];
_BufferCountSink(this._outputSink, this._count);
void add(T data) {
_buffer.add(data);
if (_buffer.length >= _count) {
_outputSink.add(List<T>.unmodifiable(_buffer));
_buffer = [];
}
}
void addError(Object error, [StackTrace? stackTrace]) {
_outputSink.addError(error, stackTrace);
}
void close() {
if (_buffer.isNotEmpty) {
_outputSink.add(List<T>.unmodifiable(_buffer));
_buffer = [];
}
_outputSink.close();
}
}
extension StreamBufferExtension<T> on Stream<T> {
Stream<List<T>> bufferCount(int count) => transform(BufferCountTransformer<T>(count));
}
6. Advanced Stream Combinators & Mappingβ
In complex applications, you frequently need to convert a stream of incoming events into asynchronous sub-tasks (like network calls or database writes). Understanding how async mapping handles concurrency is critical.
asyncMap vs switchMap vs exhaustMap vs concatMapβ
Incoming Events: ββ(Query A)βββββββ(Query B)βββββββββΊ
1. asyncMap: ββ[ Search A (300ms) ]ββββββββββββββΊ
βββ[ Search B (100ms) ]βββΊ (Race condition! B finishes before A)
2. switchMap: ββ[ Search A (Canceled!) ]ββββββββββΊ
βββ[ Search B (100ms) ]βββΊ (Cancels in-flight A, keeps latest B)
3. exhaustMap: ββ[ Search A (300ms) ]ββββββββββββββΊ
βββ(Query B Ignored!)βββββΊ (Ignores new events while busy)
4. concatMap: ββ[ Search A (300ms) ]ββββββββββββββΊ
βββ[ Search B (100ms) ]βββΊ (Sequential FIFO)
Here is a pure Dart implementation of switchMap (similar to RxDart switchMap), which cancels the previous asynchronous operation as soon as a new event arrives:
import 'dart:async';
extension SwitchMapExtension<T> on Stream<T> {
Stream<R> switchMap<R>(Future<R> Function(T event) project) {
final controller = isBroadcast
? StreamController<R>.broadcast(sync: true)
: StreamController<R>(sync: true);
int currentExecutionId = 0;
StreamSubscription<T>? subscription;
controller.onListen = () {
subscription = listen(
(data) async {
final executionId = ++currentExecutionId;
try {
final result = await project(data);
// Only emit if no newer event arrived while awaiting project(data)
if (executionId == currentExecutionId && !controller.isClosed) {
controller.add(result);
}
} catch (e, st) {
if (executionId == currentExecutionId && !controller.isClosed) {
controller.addError(e, st);
}
}
},
onError: controller.addError,
onDone: () {
if (currentExecutionId == 0) controller.close();
},
);
};
controller.onCancel = () {
subscription?.cancel();
};
return controller.stream;
}
}
Multi-Stream Synchronization: combineLatest2 in Pure Dartβ
Synchronizing two independent streams (e.g., Form Validation where isUsernameValidStream and isPasswordValidStream combine to enable the Submit button) without external dependencies:
Stream<(A, B)> combineLatest2<A, B>(Stream<A> streamA, Stream<B> streamB) {
final controller = StreamController<(A, B)>.broadcast();
A? lastA;
B? lastB;
bool hasA = false;
bool hasB = false;
StreamSubscription<A>? subA;
StreamSubscription<B>? subB;
void tryEmit() {
if (hasA && hasB && !controller.isClosed) {
controller.add((lastA as A, lastB as B));
}
}
controller.onListen = () {
subA = streamA.listen(
(a) {
lastA = a;
hasA = true;
tryEmit();
},
onError: controller.addError,
);
subB = streamB.listen(
(b) {
lastB = b;
hasB = true;
tryEmit();
},
onError: controller.addError,
);
};
controller.onCancel = () {
subA?.cancel();
subB?.cancel();
};
return controller.stream;
}
7. Subscription Hygiene & Memory Leak Preventionβ
Streams and subscriptions are among the leading causes of memory leaks in Flutter applications. When a subscription is active, the stream controller holds a reference to the subscription, which holds references to its callbacks (onData, onError), which in turn capture the enclosing class instance (State, Bloc, or Service).
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Memory Leak Chain β
β β
β StreamController / Global Event Bus β
β β holds active reference β
β βΌ β
β StreamSubscription β
β β captures lexical scope β
β βΌ β
β onData: (data) => _myStateMethod(data) β
β β captures `this` β
β βΌ β
β StatefulWidget State / Bloc (Never Garbage Collected!) β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
The Golden Rules of Stream Hygieneβ
- Every
StreamControllermust be closed: Always invokecontroller.close()when the owning object is disposed. - Every
.listen()must be cancelled: Always store theStreamSubscriptionand cancel it indispose()/ cleanup. - Never create a new Stream inside Flutter's
build()method:
// β CRITICAL ANTI-PATTERN: Recreating stream inside build()
Widget build(BuildContext context) {
return StreamBuilder<User>(
// BUG: Every widget rebuild generates a brand new stream and tears down the old one!
stream: databaseService.watchUserStream(),
builder: (context, snapshot) => Text(snapshot.data?.name ?? 'Loading...'),
);
}
// β
CORRECT: Initialize stream in initState() or use a persistent Provider/Bloc
class UserWidgetState extends State<UserWidget> {
late final Stream<User> _userStream;
void initState() {
super.initState();
_userStream = widget.databaseService.watchUserStream();
}
Widget build(BuildContext context) {
return StreamBuilder<User>(
stream: _userStream,
builder: (context, snapshot) => Text(snapshot.data?.name ?? 'Loading...'),
);
}
}
Clean Resource Disposal with CompositeSubscriptionβ
When managing multiple subscriptions in a single service or widget, cancelling them individually is error-prone. A CompositeSubscription container manages multiple subscriptions in one place:
class CompositeSubscription {
final List<StreamSubscription<dynamic>> _subscriptions = [];
bool _isDisposed = false;
bool get isDisposed => _isDisposed;
/// Adds a subscription to the tracking list
StreamSubscription<T> add<T>(StreamSubscription<T> subscription) {
if (_isDisposed) {
subscription.cancel();
throw StateError('Cannot add a subscription to a disposed CompositeSubscription.');
}
_subscriptions.add(subscription);
return subscription;
}
/// Cancels all subscriptions and clears the list
Future<void> cancelAll() async {
final futures = _subscriptions.map((s) => s.cancel()).toList();
_subscriptions.clear();
await Future.wait(futures);
}
/// Disposes the composite subscription permanently
Future<void> dispose() async {
_isDisposed = true;
await cancelAll();
}
}
extension SubscriptionDisposable<T> on StreamSubscription<T> {
void disposedBy(CompositeSubscription composite) {
composite.add(this);
}
}
Usage in a Flutter ViewModel / BLoC:β
class SearchViewModel {
final CompositeSubscription _composite = CompositeSubscription();
final SearchRepository _repository;
SearchViewModel(this._repository);
void initialize(Stream<String> searchInputQueryStream) {
searchInputQueryStream
.debounce(const Duration(milliseconds: 300))
.where((query) => query.trim().length >= 3)
.distinct()
.switchMap((query) => _repository.search(query))
.listen(_handleSearchResults)
.disposedBy(_composite); // Automatically added to composite
}
void _handleSearchResults(List<SearchResult> results) {
// Update state
}
void dispose() {
_composite.dispose(); // Cancels all subscriptions at once!
}
}
8. Summary Decision Matrix & Best Practicesβ
What is your data source?
β
ββββββββββββββββββββββββ΄βββββββββββββββββββββββ
βΌ βΌ
Single Discrete Value Continuous Sequence
(HTTP request, I/O) (Events, Sockets, UI)
β β
Use Future<T> Use Stream<T>
β
ββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββ
βΌ βΌ
Single Consumer only? Multiple Independent Listeners?
(File read, DB query) (UI State, Sensor, Chat)
β β
Single-Subscription Stream Broadcast Stream
StreamController<T>() StreamController<T>.broadcast()
β β
ββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββ
β
Is Producer Faster Than Consumer?
β
ββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββ
βΌ βΌ
YES NO
β β
Apply Backpressure Operators Standard .map()/.where()
(debounce, throttle, bufferCount)
Production Best Practices & Architectural Guidelinesβ
- π― Match Stream Semantics to Domain: Use single-subscription streams for private, sequential data producers (file I/O, database queries), and broadcast streams for multi-subscriber event buses and state notifications.
- βοΈ Leverage Cold Stream Callbacks: Implement
onListenandonCancelhooks onStreamControllerto pause sensor streams, close sockets, and release resources when there are zero active listeners. - β οΈ Avoid Synchronous Controllers: Keep
StreamController(sync: true)restricted to custom low-level transformer sinks. Default asynchronous controllers prevent subtle event order inversion and stack overflow bugs. - β±οΈ Apply Proactive Backpressure: Tame high-frequency event producers using
debounceon search inputs,throttleon UI tap interactions, andbufferCount/bufferTimefor telemetry and batch logging. - π§Ή Mandatory Subscription Hygiene: Always store
StreamSubscriptionreferences in stateful controllers and cancel them duringdispose(), or manage them with aCompositeSubscriptionto prevent severe memory leaks. - π‘οΈ Comprehensive Pipeline Error Handling: Always supply an
onErrorcallback in.listen(), or transform errors with.handleError()and.transform()to keep pipelines healthy and resilient.
With a firm grasp of stream mechanics, custom transformers, and backpressure management, you can build reactive, responsive, and robust Dart and Flutter applications capable of handling intensive asynchronous data streams with zero jank and zero memory leaks.