Token Type Measures

class tde.measures.token_type.TokenType(gold, disc, output_folder=None)

Bases: tde.measures.measures.Measure

Token Type measure

The Token measures how many ‘Gold’ token were found. The Type measures how many ‘Gold’ type were found. A ‘Type’ is a word type, and a ‘token’ is a specific occurence of a ‘type’. See https://docs.cognitive-ml.fr/tde/measures/index.html for a summary of all measures.

Input :param disc: Discovered Object, contains the discovered boundaries :param gold: Gold object, contains all the gold boundaries :param output_folder: string, path to the output folder

Output :param token_precision: Token Precision :param token_recall: Token Recall :param type_precision: Type Precision :param type_recall: Type Recall

property precision

Return Token and Type precision

property recall

Return Token and Type recall

property fscore

Return Token and Type fscore

compute_token_type()

Loop over all intervals and compute token type measure.

The Token measure is computed by counting all the gold words discovered by the system. The Type measure is computed by counting all the unique type of gold words discovered by the system.

Input: :param gold_phn: the gold phone alignment

stored as an interval tree

Parameters

• gold_wrd (Interval Tree) – the gold word alignment stored as an interval tree

• disc (list of tuples) – a list of all the discovered intervals

Output: :return: The Token Type measure

write_score()