common#

(s3prl.metric.common)

Commonly used metrics

Authors

  • Leo 2022

  • Heng-Jui Chang 2022

  • Haibin Wu 2022

accuracy#

s3prl.metric.common.accuracy(xs, ys, item_same_fn=None)[source][source]#

ter#

s3prl.metric.common.ter(hyps: List[Union[str, List[str]]], refs: List[Union[str, List[str]]]) → float[source][source]#

Token error rate calculator.

Parameters:

  • hyps (List[Union[str, List[str]]]) – List of hypotheses.

  • refs (List[Union[str, List[str]]]) – List of references.

Returns:

Averaged token error rate overall utterances.

Return type:

float

wer#

s3prl.metric.common.wer(hyps: List[str], refs: List[str]) → float[source][source]#

Word error rate calculator.

Parameters:

  • hyps (List[str]) – List of hypotheses.

  • refs (List[str]) – List of references.

Returns:

Averaged word error rate overall utterances.

Return type:

float

per#

s3prl.metric.common.per(hyps: List[str], refs: List[str]) → float[source][source]#

Phoneme error rate calculator.

Parameters:

  • hyps (List[str]) – List of hypotheses.

  • refs (List[str]) – List of references.

Returns:

Averaged phoneme error rate overall utterances.

Return type:

float

cer#

s3prl.metric.common.cer(hyps: List[str], refs: List[str]) → float[source][source]#

Character error rate calculator.

Parameters:

  • hyps (List[str]) – List of hypotheses.

  • refs (List[str]) – List of references.

Returns:

Averaged character error rate overall utterances.

Return type:

float

compute_eer#

s3prl.metric.common.compute_eer(labels: List[int], scores: List[float])[source][source]#

Compute equal error rate.

Parameters:

  • scores (List[float]) – List of hypotheses.

  • labels (List[int]) – List of references.

Returns:

Equal error rate. treshold (float): The treshold to accept a target trial.

Return type:

eer (float)

compute_minDCF#

s3prl.metric.common.compute_minDCF(labels: List[int], scores: List[float], p_target: float = 0.01, c_miss: int = 1, c_fa: int = 1)[source][source]#

Compute MinDCF. Computes the minimum of the detection cost function. The comments refer to equations in Section 3 of the NIST 2016 Speaker Recognition Evaluation Plan.

Parameters:

  • scores (List[float]) – List of hypotheses.

  • labels (List[int]) – List of references.

  • p (float) – The prior probability of positive class.

  • c_miss (int) – The cost of miss.

  • c_fa (int) – The cost of false alarm.

Returns:

The calculated min_dcf. min_c_det_threshold (float): The treshold to calculate min_dcf.

Return type:

min_dcf (float)