API reference¶
- fastabx.zerospeech_abx(item: str | Path, root: str | Path, *, max_size_group: int | None, max_x_across: int | None = _UNSET, speaker: Literal['within'] = 'within', context: Literal['within', 'any'] = 'within', distance: DistanceName = 'angular', frequency: int | str | Decimal = 50, feature_maker: Callable[[str | Path], Tensor] = torch.load, extension: str = '.pt', seed: int = 0, device: str | device | None = None, write_csv: str | Path | None = None, progress: bool = True) float[source]¶
- fastabx.zerospeech_abx(item: str | Path, root: str | Path, *, max_size_group: int | None, max_x_across: int | None, speaker: Literal['across'], context: Literal['within', 'any'] = 'within', distance: DistanceName = 'angular', frequency: int | str | Decimal = 50, feature_maker: Callable[[str | Path], Tensor] = torch.load, extension: str = '.pt', seed: int = 0, device: str | device | None = None, write_csv: str | Path | None = None, progress: bool = True) float
Compute the ABX similarly to the ZeroSpeech 2021 challenge.
On triphone or phoneme, described by an item file. Within or across speaker, and within context or ignoring context.
The item file must have the ZeroSpeech columns:
#phone,prev-phone,next-phoneandspeaker. See Item files, and Download an item file to get the ZeroSpeech item files.Returns the ABX error rate (1 - discriminability), between 0 and 1: lower is better, and chance level is 0.5.
- Parameters:
root (str | Path) – Path to the root directory containing either the features or the audio files.
max_size_group (int | None) – Maximum number of instances of A, B, or X in each
Cell. Passed to theSubsamplerof theTask. Set to 10 in the original ZeroSpeech ABX code. Required; disabled if set toNone.max_x_across (int | None) – In the “across” speaker mode, maximum number of X considered for given values of A and B. Passed to the
Subsamplerof theTask. Set to 5 in the original ZeroSpeech ABX code. Required whenspeaker="across"(passNoneexplicitly to disable it); ignored otherwise.speaker (Literal['within', 'across']) – The speaker mode, either “within” or “across”. Defaults to “within”.
context (Literal['within', 'any']) – The context mode, either “within” or “any”. Always use “within” with representations of triphones. Defaults to “within”.
distance (DistanceName) – The distance metric, “angular” (same as “cosine”), “euclidean”, “kl_symmetric” or “identical”. Defaults to “angular”.
frequency (int | str | Decimal) – The feature frequency of the features / the output of the feature maker, in Hz. Defaults to 50 Hz.
feature_maker (Callable[[str | Path], Tensor]) – Function that takes a path and returns a torch.Tensor. Defaults to
torch.load.extension (str) – The filename extension of the files to process in
root, default is “.pt”.seed (int) – The random seed for the subsampling, default is 0.
device (str | device | None) – Device on which to store the features, such as “cpu” or “cuda:1”. Defaults to CUDA if available, and CPU otherwise.
write_csv (str | Path | None) – Optional path to a CSV file where the score of every
Cellis written, as done byScore.write_csv.progress (bool) – Whether to display the progress bars while building the dataset and scoring the cells.
- Return type:
Standard classes and functions¶
Dataset¶
The from_item* constructors below all take an item file, as described in Item files.
- class fastabx.Dataset(labels, accessor)[source]¶
Simple interface to a dataset.
- Parameters:
labels (DataFrame) –
pl.DataFramecontaining the labels of the datapoints.accessor (Accessor) –
Accessorto the data, usually anInMemoryAccessor.
- classmethod from_dataframe(source, feature_columns, *, separator=',', device=None)[source]¶
Create a dataset from any tabular source containing both the labels and the features.
Accepted inputs for
source:strorPath: path to a CSV file (usesseparator).A polars or pandas
DataFrame.Mapping[str, Sequence](column name → values).Iterable[Mapping](sequence of row dictionaries).
- Parameters:
source (str | Path | DataFrame | Mapping[str, Sequence[object]] | Iterable[Mapping[str, Any]]) – The tabular source. See above for accepted types.
feature_columns (str | Collection[str]) – Column name or list of column names containing the features.
separator (str) – Separator used in the CSV file. Only relevant when
sourceis a path.device (str | device | None) – Device on which to store the features, such as “cpu” or “cuda:1”. Defaults to CUDA if available, and CPU otherwise.
- Return type:
- classmethod from_item(item, root, frequency, *, feature_maker=torch.load, extension='.pt', file_col='#file', onset_col='onset', offset_col='offset', device=None, progress=True)[source]¶
Create a dataset from an item file.
See Item files for the format of the item file, and how
#fileis matched to the feature files.If you want to keep the Libri-Light bug to reproduce previous results, set the environment variable FASTABX_WITH_LIBRILIGHT_BUG=1.
- Parameters:
root (str | Path) – Path to the root directory containing either the features or the audio files.
frequency (int | str | Decimal) – The feature frequency of the features / the output of the feature maker, in Hz. If it is not an integer, pass it as a string to avoid floating-point errors.
feature_maker (Callable[[str | Path], Tensor]) – Function that takes a path and returns a torch.Tensor. Defaults to
torch.load.extension (str) – The filename extension of the files to process in
root, default is “.pt”.file_col (str) – Column in the item file that contains the audio file names, default is “#file”.
onset_col (str) – Column in the item file that contains the onset times, default is “onset”.
offset_col (str) – Column in the item file that contains the offset times, default is “offset”.
device (str | device | None) – Device on which to store the features, such as “cpu” or “cuda:1”. Defaults to CUDA if available, and CPU otherwise.
progress (bool) – Whether to display a progress bar while building the dataset.
- Return type:
- classmethod from_item_and_units(item, units, frequency, *, audio_key='audio', units_key='units', file_col='#file', onset_col='onset', offset_col='offset', device=None, progress=True)[source]¶
Create a dataset from an item file with the units all described in a single JSONL file.
See Item files for the format of the item file.
- Parameters:
units (str | Path) – Path to the JSONL file containing the units.
frequency (int | str | Decimal) – The feature frequency, in Hz. If it is not an integer, pass it as a string to avoid floating-point errors.
audio_key (str) – Key in the JSONL file that contains the audio file names (str), default is “audio”.
units_key (str) – Key in the JSONL file that contains the units (list[int]), default is “units”.
file_col (str) – Column in the item file that contains the audio file names, default is “#file”.
onset_col (str) – Column in the item file that contains the onset times, default is “onset”.
offset_col (str) – Column in the item file that contains the offset times, default is “offset”.
device (str | device | None) – Device on which to store the features, such as “cpu” or “cuda:1”. Defaults to CUDA if available, and CPU otherwise.
progress (bool) – Whether to display a progress bar while building the dataset.
- Return type:
- classmethod from_item_with_times(item, root_features, root_times, *, feature_maker=torch.load, time_maker=torch.load, extension='.pt', file_col='#file', onset_col='onset', offset_col='offset', device=None, progress=True)[source]¶
Create a dataset from an item file.
Use arrays containing the times associated to the features instead of a given frequency. See Item files for the format of the item file.
- Parameters:
root_features (str | Path) – Path to the root directory containing either the features or the audio files.
root_times (str | Path) – Path to the root directory containing the times arrays.
feature_maker (Callable[[str | Path], Tensor]) – Function that takes a path and returns a torch.Tensor. Defaults to
torch.load.time_maker (Callable[[str | Path], Tensor]) – Function that takes a path and returns a 1D torch.Tensor. Defaults to
torch.load.extension (str) – The filename extension of the files to process in
root_featuresandroot_times, default is “.pt”.file_col (str) – Column in the item file that contains the audio file names, default is “#file”.
onset_col (str) – Column in the item file that contains the onset times, default is “onset”.
offset_col (str) – Column in the item file that contains the offset times, default is “offset”.
device (str | device | None) – Device on which to store the features, such as “cpu” or “cuda:1”. Defaults to CUDA if available, and CPU otherwise.
progress (bool) – Whether to display a progress bar while building the dataset.
- Return type:
- classmethod from_numpy(features, labels, *, device=None)[source]¶
Create a dataset from the features and the labels.
Despite the name,
featuresis not restricted to a numpy array: any input accepted bynp.asarrayworks (Python lists, tuples, CPU torch tensors via the__array__protocol, …). CUDA tensors must be moved to CPU first.- Parameters:
features (ArrayLike) – 2D array-like containing the features.
labels (DataFrame | Mapping[str, Sequence[object]]) – Dictionary of sequences, or polars/pandas DataFrame containing the labels.
device (str | device | None) – Device on which to store the features, such as “cpu” or “cuda:1”. Defaults to CUDA if available, and CPU otherwise.
- Return type:
- class fastabx.InMemoryAccessor(indices, data, device)[source]¶
Data accessor where everything is in memory.
Note
Reserved label names. Any column of Dataset.labels can be used as an ON, BY or ACROSS condition,
with one restriction: it cannot be named index, score, size, is_valid, __cell,
__group, __lookup, __pos or __triplet, and it cannot end with _a, _b or _x.
Those names are used internally when building and scoring the cells. Passing such a column to a
Task raises a ValueError: rename it beforehand. Columns not used as conditions are unaffected.
Task¶
- class fastabx.Task(dataset, *, on, by=None, across=None, subsampler=None)[source]¶
The ABX task class.
A Task builds all the
Cellgivenon,byandacrossconditions. It can be subsampled to limit the number of cells.To bypass the standard construction with a precomputed cells DataFrame, use
Task.from_cellsinstead.Every condition must be a column of
dataset.labels.- Parameters:
dataset (Dataset) – The dataset containing the features and the labels.
on (str) – The
oncondition.subsampler (Subsampler | None) – An optional subsampler to limit the number of cells and their sizes.
- classmethod from_cells(dataset, cells, *, is_symmetric)[source]¶
Build a Task from a precomputed cells DataFrame.
Use this when you have hardcoded your own triplets and want to skip the standard
on/by/acrossconstruction. The DataFrame must carry the following columns:header,description,index_a,index_b,index_x.
Subsample¶
- class fastabx.Subsampler(max_size_group, max_x_across, seed=0)[source]¶
Subsample the ABX
Task.Each cell is limited to
max_size_groupitems for A, B and X independently. When using “across” conditions, each group of (A, B) is limited tomax_x_acrosspossible values for X. Subsampling for one or more conditions can be disabled by setting the corresponding argument toNone.Note
The subsampling is reproducible given
seed, but it is not an i.i.d. sample. A single fixed seed shuffles every cell, and the same permutation is applied to all the groups of the same length. This is what keeps A and X in step in symmetric cells (where they are the same set, and where scoring relies on it to drop the diagonal), and the flip side is that the items retained in cells of equal size are correlated rather than drawn independently.- Parameters:
max_size_group (int | None) – Maximum number of instances of A, B, or X in each
Cell. Set to 10 in the original ZeroSpeech ABX code. Disabled if set toNone.max_x_across (int | None) – In the “across” speaker mode, maximum number of X considered for given values of A and B. Set to 5 in the original ZeroSpeech ABX code. Disabled if set to
None.seed (int) – The random seed for the subsampling, default is 0.
Score¶
- class fastabx.Score(task, distance_name, *, alignment='dtw', constraints=None, progress=True)[source]¶
Compute the score of a
Taskusing a given distance specified bydistance_name.All the scores reported by this class are ABX error rates (1 - discriminability). Lower is better, and chance level is 0.5.
Additional
Constraintscan be provided to restrict the possible triplets in each cell.The full scoring runs eagerly in
__init__: constructing aScoreis the expensive step, andcollapse/detailsafterwards are cheap.Warning
Constructing a
Scorewith the"cosine"/"angular"distance mutates the sharedtask.datasetin place: it L2-normalizes the features and appends the singularity-border column, so the dataset’s feature dimension grows by one andtask.dataset.accessor.is_normalizedbecomesTrue. If you need the original features back, keep a separate, un-normalizedDataset.- Parameters:
distance_name (DistanceName | Distance) – The distance to use, either the name of a built-in one (“euclidean”, “cosine”, “angular”, “kl_symmetric”, “identical”) or a custom
Distancecallable.alignment (AlignmentName | Alignment) – How to reduce the frame-level cost lattice to one distance per pair of sequences, either the name of a built-in alignment (“dtw”) or a custom
Alignment. Defaults to “dtw”. Bypassed entirely when the dataset is pooled, since there is nothing to align.constraints (Constraints | None) – Optional constraints to restrict the possible triplets.
progress (bool) – Whether to display a progress bar while scoring the cells.
See Alignment to change how sequences spanning several frames are compared.
- property cells: DataFrame[source]¶
Scored cells.
The
scorecolumn is the ABX error rate of each cell, andsizeits number of triplets.
- collapse(*, levels=None, weighted=False)[source]¶
Collapse the scored cells into the final ABX error rate.
Use either levels or weighted=True to collapse the scores.
Advanced¶
Pooling¶
Pooling collapses the frame-level features of each token into a single vector, so that every token is
represented by one fixed-size embedding instead of a variable-length sequence. This is useful when you
want token-level (rather than frame-level) representations: the comparison no longer needs an
alignment, which makes the distance computation faster. Two methods are available:
"mean" averages the frames, and "hamming" averages them using a Hamming window (giving less
weight to the boundary frames).
- fastabx.pool_dataset(dataset, pooling_name)[source]¶
Pool the
Datasetusing the pooling method given bypooling_name.The pooled dataset is a new one, with data stored in memory on the same device as
dataset. For simplicity, we iterate through the original dataset and apply pooling on each element.- Parameters:
dataset (Dataset) – The dataset to pool.
pooling_name (PoolingName) – The pooling method, either “mean” or “hamming”.
- Return type:
- class fastabx.PooledDataset(labels, accessor, pooling)[source]¶
Pooled dataset.
- Parameters:
labels (DataFrame)
accessor (Accessor)
pooling (PoolingName)
- class fastabx.PoolingName¶
Type alias for
Literal["mean", "hamming"].
Cell¶
- class fastabx.Cell(a, b, x, header, description, is_symmetric)[source]¶
Individual cell of the ABX task.
Cells are the unit of work for the ABX
TaskandScore. They are collections of triplets (A, B, X) that share the same values for theon,byandacrossconditions.- Parameters:
Accessor¶
- class fastabx.Accessor[source]¶
How the ABX pipeline reads the features of a
Dataset.InMemoryAccessoris the only implementation shipped by fastabx, and everything it needs fits in memory. Anything satisfying this protocol works in its place: a memory-mapped store, a lazy reader, an accessor keeping its data on a device of its own. The pipeline only ever reads throughlengthsandbatched, so those two are the ones that must be fast.Indices are the row numbers of
Dataset.labels: itemiof the accessor describes rowi.- batched(indices)[source]¶
Gather the given datapoints into a single padded
Batch.- Parameters:
indices (ArrayLike) – The indices of the datapoints. The order of the batch follows the order given here.
- Return type:
Distance¶
- fastabx.abx_on_cell(cell, distance_name='angular', *, alignment=dtw_batch)[source]¶
Compute the ABX of a
cellusing the givendistance.Returns the ABX error rate (1 - discriminability) of the cell, as a scalar tensor.
Warning
Unlike
Score, this low-level helper does not normalize the features. For the default"angular"(and"cosine") distance the cell’s features must already be L2-normalized (e.g. viaDataset.normalize_); otherwise the dot products are only clamped to[-1, 1]and the score is silently wrong. Likewise"kl_symmetric"expects the features to be probability distributions.- Parameters:
cell (Cell) – The cell to compute the ABX on.
distance_name (DistanceName | Distance) – The distance to use, either the name of a built-in one (“euclidean”, “cosine”, “angular”, “kl_symmetric”, “identical”) or a custom
Distancecallable. Defaults to “angular”.alignment (Alignment) – How to align sequences that span several frames, as an
Alignmentcallable. Defaults totorchdtw.dtw_batch. Never called on the distance matrices whose lattice is1x1.
- Return type:
- class fastabx.DistanceName¶
Type alias for
Literal["euclidean", "cosine", "angular", "kl_symmetric", "identical"]."cosine"is an alias for"angular".
- class fastabx.Distance¶
Type alias for
Callable[[torch.Tensor, torch.Tensor], torch.Tensor]: a function taking two batches of representations and returning their pairwise frame-level distances, as a(n1, n2, s1, s2)cost lattice. Reducing that lattice to one distance per pair of sequences is the job of an alignment.Anywhere a
DistanceNameis accepted (Scoreandabx_on_cell) a callable of this shape is accepted in its place. Only the built-in"angular"and"cosine"names normalize the dataset, so a custom distance is handed the features exactly as they are.
Alignment¶
A Distance compares individual frames while an Alignment turns the resulting (n1, n2, s1, s2)
cost lattice into the (n1, n2) distance between the sequences themselves. Dynamic time warping is the
default. When every sample has a single frame (a pooled dataset, see Pooling), the alignment is bypassed and the
frame cost is used directly.
- class fastabx.AlignmentName¶
Type alias for
Literal["dtw"], the only alignment available for now.
- class fastabx.Alignment[source]¶
Reduce a frame-level distance lattice to one distance per pair of sequences.
Implementations must return the distance normalized by the length of the alignment path, so that pairs of different lengths stay comparable: the ABX decision compares a X-to-A distance against a X-to-B distance, and an unnormalized distance would bias it towards the shorter pair.
Only the
(sx[i], sy[j])sub-block of each pair may be read; anything beyond those lengths is padding.Anywhere an
AlignmentNameis accepted, a custom callable satisfying this protocol is accepted too. The lattice and the two length tensors are passed positionally, so an implementation may name them freely. Extension entry-point for alignments that fastabx does not ship, such as an edit distance:def edit(cost: Tensor, sx: Tensor, sy: Tensor, *, symmetric: bool) -> Tensor: ... # your dynamic program over the lattice Score(task, "identical", alignment=edit)
- __call__(distances, sx, sy, /, *, symmetric)[source]¶
Align every pair of sequences.
The three tensors are positional-only: an implementation is free to name them whatever suits it, but
symmetricis passed by keyword and must keep its name.- Parameters:
- Returns:
A
(n1, n2)tensor of sequence distances.- Return type:
Constraints¶
- class fastabx.Constraints¶
Type alias for
Iterable[pl.Expr].See With constraints to understand how to use them.
- fastabx.constraints_all_different(*columns)[source]¶
Return
Constraintsthat ensure that each specified column has different values for A, B and X.- Parameters:
columns (str) – The columns to apply the constraints on.
- Return type:
Environment variables¶
Behaviour¶
FASTABX_WITH_LIBRILIGHT_BUG: If set to 1, changes the behaviour ofDataset.from_itemto match Libri-Light. Every feature will now be one frame shorter. This should be set only if you want to replicate previous results obtained with Libri-Light / ZeroSpeech 2021. See Slicing features for more details on how features are sliced.TQDM_DISABLE: If set, every fastabx progress bar is hidden, overridingprogressarguments and--quietflags.
Performance tuning¶
The variables below bound the size of the intermediate tensors in the scoring engine. Normal usage should not require changing them: lower them if the scoring runs out of memory, raise them if you have memory to spare and the cells are small.
FASTABX_MAX_SCORE_CHUNK_ROWS(default 8192): Maximum number of rows compared at once when scoring a group of cells. Turn down if you have an out-of-memory error.FASTABX_GATHER_CHUNK_ROWS(default 8192): Maximum number of rows gathered and padded in a single batched read from theInMemoryAccessor.FASTABX_REDUCTION_FLUSH_COLS(default 262144): Number of accumulated columns after which the per-cell reduction is flushed. Larger values amortise the reduction over more cells, at the cost of keeping more intermediate counts around.
Exceptions¶
Building a Dataset¶
- exception fastabx.FrequencyTypeError[source]¶
If frequency is of a type that can lead to floating-point unexpected behavior.
- Return type:
None
- exception fastabx.FeaturesSizeError(fileid, start, end, actual)[source]¶
To raise if the features size is not correct.
- exception fastabx.EmptyFeaturesError(df)[source]¶
Raised when empty features are found when building the dataset.
- Parameters:
df (DataFrame)
- Return type:
None
- exception fastabx.EmptyDatasetError[source]¶
The dataset holds no datapoint at all.
- Return type:
None
- exception fastabx.NonContiguousIndicesError(count, lowest, highest)[source]¶
The accessor indices are not exactly
range(len(indices)).
- exception fastabx.NonFiniteError(fileid)[source]¶
To raise if non-finite features have been found.
- Parameters:
fileid (str)
- Return type:
None
Building a Task¶
- exception fastabx.EmptyTaskError(on, by, across)[source]¶
No cell could be built for the given conditions.
- exception fastabx.LabelReservedNameError(name)[source]¶
Invalid name for a condition.
- Parameters:
name (str)
- Return type:
None
- exception fastabx.LabelSuffixError(name)[source]¶
Invalid suffix for a condition.
- Parameters:
name (str)
- Return type:
None
- exception fastabx.UnknownConditionError(missing, available)[source]¶
A condition is not a column of
Dataset.labels.
Scoring¶
- exception fastabx.CollapseError(*, are_set, conditions=())[source]¶
Something wrong happened when collapsing the
Score.
- exception fastabx.EmptyScoreError[source]¶
Every cell has a null score, so there is nothing left to average.
- Return type:
None
- exception fastabx.IdenticalDistanceDimensionError(dim)[source]¶
The “identical” distance got features with more than one dimension.
- Parameters:
dim (int)
- Return type:
None
- exception fastabx.IncompatibleNormalizationError(distance_name)[source]¶
The dataset was already L2-normalized for a previous angular score and cannot be reused for this distance.
- Parameters:
distance_name (str)
- Return type:
None