flair.datasets.biomedical.HUNER_ALL_BIOID

class flair.datasets.biomedical.HUNER_ALL_BIOID(*args, **kwargs)View on GitHub

Bases: BIGBIO_NER_CORPUS

__init__(*args, **kwargs)View on GitHub

Initialize the BigBio Corpus.

Parameters:

• dataset_name – Name of the dataset in the huggingface hub (e.g. nlmchem or bigbio/nlmchem)

• base_path – Path to the corpus on your machine

• in_memory – If True, keeps dataset in memory giving speedups in training.

• sentence_splitter – Custom implementation of SentenceSplitter which segments the text into sentences and tokens (default SciSpacySentenceSplitter)

• train_split_name – Name of the training split in bigbio, usually train (default: None)

• dev_split_name – Name of the development split in bigbio, usually validation (default: None)

• test_split_name – Name of the test split in bigbio, usually test (default: None)

Methods

__init__(*args, **kwargs)	Initialize the BigBio Corpus.
add_label_noise(label_type, labels[, ...])	Generates uniform label noise distribution in the chosen dataset split.
bin_search_passage(passages, low, high, entity)	Helper methods to find the passage to a given entity mention (incl.
build_corpus_directory_name(dataset_name)	Builds the directory name for the given data set.
downsample([percentage, downsample_train, ...])	Randomly downsample the corpus to the given percentage (by removing data points).
filter_empty_sentences()	A method that filters all sentences consisting of 0 tokens.
filter_long_sentences(max_charlength)	A method that filters all sentences for which the plain text is longer than a specified number of characters.
get_all_sentences()	Returns all sentences (spanning all three splits) in the Corpus.
get_entity_type_mapping()	Return the mapping of entity type given in the dataset to canonical types.
get_label_distribution()	Counts occurrences of each label in the corpus and returns them as a dictionary object.
make_label_dictionary(label_type[, ...])	Creates a dictionary of all labels assigned to the sentences in the corpus.
make_tag_dictionary(tag_type)	Create a tag dictionary of a given label type.
make_vocab_dictionary([max_tokens, min_freq])	Creates a Dictionary of all tokens contained in the corpus.
obtain_statistics([label_type, pretty_print])	Print statistics about the corpus, including the length of the sentences and the labels in the corpus.
to_internal_dataset(dataset, split)	Converts a dataset given in hugging datasets format to our internal corpus representation.

Attributes

dev	The dev split as a torch.utils.data.Dataset object.
test	The test split as a torch.utils.data.Dataset object.
train	The training split as a torch.utils.data.Dataset object.

get_entity_type_mapping()View on GitHub

Return the mapping of entity type given in the dataset to canonical types.

Note, if a entity type is not present in the map it is discarded.

Return type:

Optional[dict]

build_corpus_directory_name(dataset_name)View on GitHub

Builds the directory name for the given data set.

Return type:

str