interface#

(s3prl.nn.interface)

Model interfaces

Authors:

  • Leo 2022

AbsUpstream#

class s3prl.nn.interface.AbsUpstream(*args, **kwargs)[source][source]#

Bases: Module

The upstream model should follow this interface. Please subclass it.

property num_layer: int[source]#

number of hidden states

property hidden_sizes: List[int][source]#

hidden size of each hidden state

property downsample_rates: List[int][source]#

downsample rate from 16 KHz waveforms for each hidden state

forward(wavs: FloatTensor, wavs_len: LongTensor) → Tuple[List[FloatTensor], List[LongTensor]][source][source]#
Parameters:

  • wavs (torch.FloatTensor) – (batch_size, seq_len, 1)

  • wavs_len (torch.LongTensor) – (batch_size, )

Returns:

  1. all_hs (List[torch.FloatTensor]): all the hidden states

  2. all_hs_len (List[torch.LongTensor]): the lengths for all the hidden states

Return type:

tuple

call_super_init: bool = False[source]#
dump_patches: bool = False[source]#
training: bool[source]#

AbsFeaturizer#

class s3prl.nn.interface.AbsFeaturizer(*args, **kwargs)[source][source]#

Bases: Module

The featurizer should follow this interface. Please subclass it. The featurizer’s mission is to reduce (standardize) the multiple hidden states from AbsUpstream into a single hidden state, so that the downstream model can use it as a conventional representation.

property output_size: int[source]#

The output size after hidden states reduction

property downsample_rate: int[source]#

The downsample rate from 16 KHz waveform of the reduced single hidden state

forward(all_hs: List[FloatTensor], all_hs_len: List[LongTensor]) → Tuple[FloatTensor, LongTensor][source][source]#
Parameters:

  • all_hs (List[torch.FloatTensor]) – all the hidden states

  • all_hs_len (List[torch.LongTensor]) – the lengths for all the hidden states

Returns:

  1. hs (torch.FloatTensor)

  2. hs_len (torch.LongTensor)

Return type:

tuple

call_super_init: bool = False[source]#
dump_patches: bool = False[source]#
training: bool[source]#

AbsFrameModel#

class s3prl.nn.interface.AbsFrameModel(*args, **kwargs)[source][source]#

Bases: Module

The frame-level model interface.

property input_size: int[source]#
property output_size: int[source]#
forward(x: FloatTensor, x_len: LongTensor) → Tuple[FloatTensor, LongTensor][source][source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

call_super_init: bool = False[source]#
dump_patches: bool = False[source]#
training: bool[source]#

AbsUtteranceModel#

class s3prl.nn.interface.AbsUtteranceModel(*args, **kwargs)[source][source]#

Bases: Module

The utterance-level model interface, which pools the temporal dimension.

property input_size: int[source]#
property output_size: int[source]#
forward(x: FloatTensor, x_len: LongTensor) → FloatTensor[source][source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

call_super_init: bool = False[source]#
dump_patches: bool = False[source]#
training: bool[source]#