flair.data.Token#

class flair.data.Token(text, head_id=None, whitespace_after=1, start_position=0, sentence=None)View on GitHub#

Bases: _PartOfSentence

This class represents one word in a tokenized sentence.

Each token may have any number of tags. It may also point to its head in a dependency tree.

__init__(text, head_id=None, whitespace_after=1, start_position=0, sentence=None)View on GitHub#

Methods

__init__(text[, head_id, whitespace_after, ...])

add_label(typename, value[, score])

Adds a label to the DataPoint by internally creating a Label object.

add_metadata(key, value)

add_tags_proba_dist(tag_type, tags)

clear_embeddings([embedding_names])

get_each_embedding([embedding_names])

get_embedding([names])

get_head()

get_label([label_type, zero_tag_value])

get_labels([typename])

Returns all labels of this datapoint belonging to a specific annotation layer.

get_metadata(key)

get_tags_proba_dist(tag_type)

has_label(type)

has_metadata(key)

remove_labels(typename)

set_embedding(name, vector)

set_label(typename, value[, score])

to(device[, pin_memory])

to_dict([tag_type])

Attributes

embedding

end_position

idx

labels

score

start_position

tag

text

unlabeled_identifier

property idx: int#
property text: str#
property unlabeled_identifier: str#
add_tags_proba_dist(tag_type, tags)View on GitHub#
Return type:

None

get_tags_proba_dist(tag_type)View on GitHub#
Return type:

list[Label]

get_head()View on GitHub#
property start_position: int#
property end_position: int#
property embedding#
add_label(typename, value, score=1.0, **metadata)View on GitHub#

Adds a label to the DataPoint by internally creating a Label object.

Parameters:

  • typename (str) – A string that identifies the layer of annotation, such as “ner” for named entity labels or “sentiment” for sentiment labels

  • value (str) – A string that sets the value of the label.

  • score (float) – Optional value setting the confidence level of the label (between 0 and 1). If not set, a default confidence of 1 is used.

  • **metadata – Additional metadata information.

Returns:

A pointer to itself (DataPoint object, now with an added label).

set_label(typename, value, score=1.0, **metadata)View on GitHub#
to_dict(tag_type=None)View on GitHub#
Return type:

dict[str, Any]