flair.datasets.sequence_labeling.NER_MULTI_CONER_V2

class flair.datasets.sequence_labeling.NER_MULTI_CONER_V2(task='multi', base_path=None, in_memory=True, use_dev_as_test=True, **corpusargs)View on GitHub

Bases: MultiFileColumnCorpus

__init__(task='multi', base_path=None, in_memory=True, use_dev_as_test=True, **corpusargs)View on GitHub

Initialize the MultiCoNer V2 corpus for the Semeval2023 workshop.

This is only possible if you’ve applied and downloaded it to your machine. Apply for the corpus from here https://multiconer.github.io/dataset and unpack the .zip file’s content into a folder called ‘ner_multi_coner_v2’. Then set the base_path parameter in the constructor to the path to the parent directory where the ner_multi_coner_v2 folder resides. You can also create the multiconer in the {FLAIR_CACHE_ROOT}/datasets folder to leave the path empty. :type base_path: Union[str, Path, None] :param base_path: Path to the ner_multi_coner_v2 corpus (i.e. ‘ner_multi_coner_v2’ folder) on your machine POS tags or chunks respectively :type in_memory: bool :param in_memory: If True, keeps dataset in memory giving speedups in training. :type use_dev_as_test: bool :param use_dev_as_test: If True, it uses the dev set as test set and samples random training data for a dev split. :type task: str :param task: either ‘multi’, ‘code-switch’, or the language code for one of the mono tasks.

Methods

__init__([task, base_path, in_memory, ...])	Initialize the MultiCoNer V2 corpus for the Semeval2023 workshop.
add_label_noise(label_type, labels[, ...])	Generates uniform label noise distribution in the chosen dataset split.
downsample([percentage, downsample_train, ...])	Randomly downsample the corpus to the given percentage (by removing data points).
filter_empty_sentences()	A method that filters all sentences consisting of 0 tokens.
filter_long_sentences(max_charlength)	A method that filters all sentences for which the plain text is longer than a specified number of characters.
get_all_sentences()	Returns all sentences (spanning all three splits) in the Corpus.
get_label_distribution()	Counts occurrences of each label in the corpus and returns them as a dictionary object.
make_label_dictionary(label_type[, ...])	Creates a dictionary of all labels assigned to the sentences in the corpus.
make_tag_dictionary(tag_type)	Create a tag dictionary of a given label type.
make_vocab_dictionary([max_tokens, min_freq])	Creates a Dictionary of all tokens contained in the corpus.
obtain_statistics([label_type, pretty_print])	Print statistics about the corpus, including the length of the sentences and the labels in the corpus.

Attributes

dev	The dev split as a torch.utils.data.Dataset object.
test	The test split as a torch.utils.data.Dataset object.
train	The training split as a torch.utils.data.Dataset object.