flair.tokenization.SpaceTokenizer

class flair.tokenization.SpaceTokenizerView on GitHub

Bases: Tokenizer

Tokenizer based on space character only.

__init__()View on GitHub

Methods

__init__()
run_tokenize(text)
tokenize(text)

Attributes

name

tokenize(text)View on GitHub

Return type:

list[str]

static run_tokenize(text)View on GitHub

Return type:

list[str]