Chunking Strings in Dart: Every Practical Approach from Regex to Unicode-Safe Splitting
String chunking is a common task in Dart and Flutter applications. You frequently need to split a long string into fixed-size pieces for:
- Text formatting and line wrapping
- Creating preview snippets or terminal output
- Pagination and UI layouts
- Network payloads and protocol batching
- Processing international text and emojis
- Chunking streams or database records
For example, given:
const text = 'HelloWorld';
const size = 3;
we want to split text into slices of length 3:
Hel
loW
orl
d
Dart does not provide a built-in chunk() method on String, but there are several ways to implement it. The best approach depends on how your application defines a "character": UTF-16 code units, Unicode code points, grapheme clusters, or raw UTF-8 bytes.
This guide covers every practical chunking technique in Dart, their performance trade-offs, and when to use each.
1. What Does "Chunking" Mean in Dart?โ
Before choosing an algorithm, define the unit of measurement for chunk size. Dart strings can be measured in four distinct ways:
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ "Hello ๐จโ๐ฉโ๐งโ๐ฆ" โ
โโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Unit โ Meaning โ
โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ UTF-16 code units โ Dart's default String indexing model โ
โ Unicode runes โ Individual Unicode code points โ
โ Grapheme clusters โ Visual, user-perceived characters โ
โ UTF-8 bytes โ Encoded byte length for network/storage โ
โโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
For plain ASCII strings like 'HelloWorld', all four units yield identical results. However, with emojis (e.g., ๐ or family sequences ๐จโ๐ฉโ๐งโ๐ฆ) and combining accents (e.g., eฬ), index-based slicing can split multibyte sequences into corrupted characters.
2. Technique 1: Standard Loop with substring() (Best Default)โ
For machine data, ASCII text, and standard string processing, an imperative for loop using substring() is the fastest and most readable solution.
Basic Implementationโ
List<String> chunkString(String text, int size) {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
final chunks = <String>[];
for (var i = 0; i < text.length; i += size) {
final end = (i + size < text.length) ? i + size : text.length;
chunks.add(text.substring(i, end));
}
return chunks;
}
Usage:
void main() {
final chunks = chunkString('HelloWorld', 3);
print(chunks); // [Hel, loW, orl, d]
}
Extension Methodโ
To create a fluent, reusable API across your project, wrap the logic in a String extension:
extension StringChunking on String {
List<String> chunk(int size) {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
final result = <String>[];
for (var i = 0; i < length; i += size) {
final end = (i + size).clamp(0, length);
result.add(substring(i, end));
}
return result;
}
}
// Usage:
final pieces = 'The quick brown fox'.chunk(5);
Pros:
- Zero external dependencies
- Fast and memory-efficient
- Clean and readable
Cons:
- Operates on UTF-16 code units; may split surrogate pairs if applied to non-BMP characters.
3. Technique 2: Lazy Generator with sync* (Best for Large Strings)โ
When processing large strings or streaming chunks to another service, you don't need to allocate the entire List<String> in memory upfront. A sync* generator yields chunks on demand.
Iterable<String> chunkStringLazy(String text, int size) sync* {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
for (var i = 0; i < text.length; i += size) {
final end = (i + size).clamp(0, text.length);
yield text.substring(i, end);
}
}
Usage:
void main() {
// Only computes each chunk as the loop iterates
for (final chunk in chunkStringLazy(largeLogFile, 1024)) {
sendToServer(chunk);
}
}
Pros:
- Minimal memory footprint for large datasets
- Composable with standard
Iterablemethods (map,take,where)
4. Technique 3: Unicode Code Points with runesโ
Dart's runes getter exposes full 32-bit Unicode code points. This prevents splitting surrogate pairs (such as ๐, which uses 2 UTF-16 code units).
List<String> chunkByRunes(String text, int size) {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
final runes = text.runes.toList();
final chunks = <String>[];
for (var i = 0; i < runes.length; i += size) {
final end = (i + size).clamp(0, runes.length);
chunks.add(String.fromCharCodes(runes.sublist(i, end)));
}
return chunks;
}
Lazy rune chunker:
Iterable<String> chunkByRunesLazy(String text, int size) sync* {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
var buffer = <int>[];
for (final rune in text.runes) {
buffer.add(rune);
if (buffer.length == size) {
yield String.fromCharCodes(buffer);
buffer = <int>[];
}
}
if (buffer.isNotEmpty) {
yield String.fromCharCodes(buffer);
}
}
Note: While
runessolves individual code points, complex characters like๐จโ๐ฉโ๐งโ๐ฆ(composed of multiple code points joined by Zero-Width Joiners) or accented letters (e.g.e+ combining accent) can still be divided. For user-facing text, use grapheme clusters.
5. Technique 4: Grapheme Clusters with characters (Best for User-Facing UI)โ
For user-visible text, emojis, and international strings, use the official characters package to operate on extended grapheme clusters (visual characters).
Add the dependency:
dependencies:
characters: ^1.4.0
Eager List Chunkingโ
import 'package:characters/characters.dart';
List<String> chunkByCharacters(String text, int size) {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
final chunks = <String>[];
var buffer = StringBuffer();
var count = 0;
for (final char in text.characters) {
buffer.write(char);
count++;
if (count == size) {
chunks.add(buffer.toString());
buffer.clear();
count = 0;
}
}
if (buffer.isNotEmpty) {
chunks.add(buffer.toString());
}
return chunks;
}
Lazy Grapheme Chunkingโ
import 'package:characters/characters.dart';
Iterable<String> chunkByCharactersLazy(String text, int size) sync* {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
var buffer = StringBuffer();
var count = 0;
for (final char in text.characters) {
buffer.write(char);
count++;
if (count == size) {
yield buffer.toString();
buffer = StringBuffer();
count = 0;
}
}
if (count > 0) {
yield buffer.toString();
}
}
Usage:
final text = 'Family: ๐จโ๐ฉโ๐งโ๐ฆ and flags: ๐บ๐ธ';
final chunks = chunkByCharacters(text, 2);
// Result safely preserves complex emojis without visual corruption!
6. Technique 5: Chunking by UTF-8 Bytes (Network & Protocols)โ
When interacting with binary protocols, databases, or rate-limited network APIs, the constraint is often byte size, not character count.
Raw Byte Chunksโ
import 'dart:convert';
List<List<int>> chunkBytes(String text, int maxBytes) {
if (maxBytes <= 0) {
throw ArgumentError.value(maxBytes, 'maxBytes', 'Byte size must be greater than zero.');
}
final bytes = utf8.encode(text);
final chunks = <List<int>>[];
for (var i = 0; i < bytes.length; i += maxBytes) {
final end = (i + maxBytes).clamp(0, bytes.length);
chunks.add(bytes.sublist(i, end));
}
return chunks;
}
Text Chunking within a Byte Budgetโ
If each resulting chunk must remain valid, decodeable UTF-8 text while staying under a maximum byte limit:
import 'dart:convert';
Iterable<String> chunkUtf8Text(String text, int maxBytes) sync* {
if (maxBytes <= 0) {
throw ArgumentError.value(maxBytes, 'maxBytes', 'Byte limit must be greater than zero.');
}
var buffer = StringBuffer();
var currentByteLength = 0;
for (final rune in text.runes) {
final char = String.fromCharCode(rune);
final charBytes = utf8.encode(char).length;
if (buffer.isNotEmpty && currentByteLength + charBytes > maxBytes) {
yield buffer.toString();
buffer = StringBuffer();
currentByteLength = 0;
}
buffer.write(char);
currentByteLength += charBytes;
}
if (buffer.isNotEmpty) {
yield buffer.toString();
}
}
7. Technique 6: Regex-Based Chunkingโ
Regex offers a concise one-liner for chunking. Use the dotAll: true flag so that newlines are treated as matching characters:
List<String> chunkWithRegex(String text, int size) {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
final regex = RegExp('.{1,$size}', dotAll: true);
return regex.allMatches(text).map((m) => m.group(0)!).toList();
}
When to use:
- When chunking is part of a broader regular expression pattern match.
- For quick scripting where regex is already imported.
8. Technique 7: Generic Iterable Chunkingโ
Chunking is not unique to strings. A generic chunk() function can slice any Iterable<T>:
Iterable<List<T>> chunkIterable<T>(Iterable<T> items, int size) sync* {
if (size <= 0) {
throw ArgumentError.value(size, 'size', 'Chunk size must be greater than zero.');
}
var chunk = <T>[];
for (final item in items) {
chunk.add(item);
if (chunk.length == size) {
yield chunk;
chunk = <T>[];
}
}
if (chunk.isNotEmpty) {
yield chunk;
}
}
Usage:
// Chunk numbers, models, or DB rows
final batches = chunkIterable([1, 2, 3, 4, 5, 6, 7], 3);
print(batches.toList()); // [[1, 2, 3], [4, 5, 6], [7]]
// Use with runes or characters
final runeChunks = chunkIterable(text.runes, 3).map(String.fromCharCodes);
9. Alternative Approaches & When to Avoid Themโ
| Approach | Code Sample | Recommendation |
|---|---|---|
List.generate() | List.generate((text.length / size).ceil(), (i) => text.substring(i * size, ...)) | โ
Declarative, but slightly less readable than a for loop. |
split('') | text.split('') | โ Inefficient. Slices individual characters into single-letter lists before re-joining. |
| Recursion | [text.substring(0, end), ...chunk(text.substring(end))] | โ High stack overhead and intermediate string allocations. |
skip() and take() | characters.skip(i).take(size) in a loop | โ O(Nยฒ) quadratic overhead due to repeated iterator traversals. |
10. Edge Cases & Input Validationโ
Ensure your chunking utility handles standard boundary conditions gracefully:
- Non-Positive Size (
size <= 0): Always throw anArgumentError. Never allowsize = 0to enter an infinite loop. - Empty String (
''): Should return[](empty collection), not['']. - Size Greater than String Length: For
'Hi'withsize = 10, return['Hi']. - Non-Uniform Chunks: If
text.length % size != 0, the trailing chunk contains the remaining characters.
11. Unit Testing Your Chunking Implementationโ
Here is a comprehensive unit test suite using package:test covering all common edge cases, ASCII, newlines, and Unicode:
import 'package:test/test.dart';
void main() {
group('chunkString tests', () {
test('chunks exact multiple evenly', () {
expect(chunkString('abcdef', 2), equals(['ab', 'cd', 'ef']));
});
test('handles remainder on trailing chunk', () {
expect(chunkString('abcdefg', 2), equals(['ab', 'cd', 'ef', 'g']));
});
test('returns single item when size exceeds length', () {
expect(chunkString('abc', 10), equals(['abc']));
});
test('returns empty list for empty string', () {
expect(chunkString('', 3), isEmpty);
});
test('handles size = 1', () {
expect(chunkString('abc', 1), equals(['a', 'b', 'c']));
});
test('throws ArgumentError on size <= 0', () {
expect(() => chunkString('abc', 0), throwsArgumentError);
expect(() => chunkString('abc', -1), throwsArgumentError);
});
test('preserves multi-line text with newlines', () {
expect(chunkString('hello\nworld', 4), equals(['hell', 'o\nwo', 'rld']));
});
});
group('Unicode & Grapheme tests', () {
test('chunkByCharacters preserves emojis without corruption', () {
final text = '๐๐๐';
expect(chunkByCharacters(text, 2), equals(['๐๐', '๐']));
});
});
}
12. Decision Matrix & Summaryโ
Choose your chunking strategy based on your data type and performance requirements:
| Use Case | Recommended Method | Dependency | Lazy Support |
|---|---|---|---|
| Standard ASCII / General Strings | substring() in a for loop | None | sync* |
| Large Files / Memory Sensitive | sync* Generator | None | Native |
| UI, Emojis & International Text | text.characters loop | characters | sync* |
| Unicode Code Points (Surrogate Pairs) | text.runes | None | sync* |
| Network Payloads / Byte Limits | utf8.encode() byte chunks | dart:convert | sync* |
| General Collections / Batches | Generic chunkIterable<T> | None | sync* |
What are you chunking?
โ
โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโ
โผ โผ โผ
Plain Text / Logs UI / Emojis Binary / Network
โ โ โ
substring() Loop text.characters utf8.encode()
(or sync* lazy) (package:chars) (byte chunks)
By choosing the right character unit upfront, you ensure predictable behavior, optimal performance, and robust Unicode support across all Flutter and Dart applications.