NeMo / docs /source /common /tokenizers.rst
camenduru's picture
thanks to NVIDIA ❤
7934b29
raw
history blame contribute delete
316 Bytes
Tokenizers
----------
.. autoclass:: nemo.collections.common.tokenizers.AutoTokenizer
:special-members: __init__
.. autoclass:: nemo.collections.common.tokenizers.SentencePieceTokenizer
:special-members: __init__
.. autoclass:: nemo.collections.common.tokenizers.TokenizerSpec
:special-members: __init__