Performance and memory¶
fastabx keeps the dataset in memory on a single device. Scoring intermediates are chunked. This page describes what is allocated, when, and which knob to turn when it does not fit.
Where the data lives¶
A Dataset loads all the features at once, into a single tensor on one device: CUDA if a GPU is
visible, CPU otherwise. Every constructor takes a device argument to choose another one.
from fastabx import Dataset
dataset = Dataset.from_item(item, features, 50) # CUDA if available
dataset = Dataset.from_item(item, features, 50, device="cpu") # forced on CPU
dataset = Dataset.from_item(item, features, 50, device="cuda:1") # second GPU
The tensor holds every token, concatenated along time: its size is
total number of frames × dimension × element size (4 bytes in float32, 8 in float64). Constructors preserve
the source dtype unless dtype=... requests a conversion. A corpus of a few hundred
thousand triphones, at 50 Hz and 768 dimensions, corresponds to a few gigabytes.
Ten times that is probably too large for your GPU: the fix is either device="cpu", or pooling (see below).
One thing to add: the "angular" and "cosine" distances L2-normalize the dataset in place and append a
singularity column, so the tensor is rewritten one dimension wider.
While scoring¶
Cells are not scored one by one. Cells that share the same X and A sets are gathered into a group and scored
together, so each pair of sequences is compared once instead of once per cell it appears in. Within a group,
the cost of a comparison is a frame-level lattice: comparing n sequences of at most s frames
against m sequences of at most t frames allocates n × m × s × t floats, which an
alignment then reduces to one distance per pair.
That product may lead to running out of memory. It is bounded by two environment variables described in Performance tuning:
FASTABX_MAX_SCORE_CHUNK_ROWScaps how many sequences are compared against the group’s X at once. Lower it first on an out-of-memory error.FASTABX_GATHER_CHUNK_ROWScaps how many rows are gathered and padded in one read.FASTABX_REDUCTION_FLUSH_COLScaps how many reduced contribution columns are retained before a flush.
What dominates the runtime¶
Three quantities, in decreasing order of how often they matter:
The number of triplets. It grows with the product of the cell sizes, so a handful of large cells can cost
more than thousands of small ones. This is what the Subsampler is for:
from fastabx import Subsampler, Task
subsampler = Subsampler(max_size_group=10, max_x_across=5)
task = Task(dataset, on="#phone", by=["prev-phone", "next-phone"], across=["speaker"], subsampler=subsampler)
The sequence lengths. The lattice is quadratic in them, and dynamic time warping then runs over it.
The alignment. DTW is an efficient C++ extension but it is still work, which can be skipped via pooling.
Pooling¶
pool_dataset collapses each token to a single vector, so every sequence has one frame:
from fastabx import pool_dataset
pooled = pool_dataset(dataset, "mean")
When it is acceptable for your evaluation, it is the single biggest speedup available.
Progress and measurement¶
Both long phases show a progress bar: building the dataset, and scoring the cells. Pass progress=False to
the Dataset constructors, to Score, or to zerospeech_abx to silence them in a
pipeline; the CLI takes --quiet. Setting TQDM_DISABLE hides every bar whatever the arguments say.
Building the Score is where the computation happens; it runs eagerly in the constructor.
Score.collapse and Score.details only aggregate numbers that already exist.