hello.fiftyone.brain module

hello.fiftyone.brain.classification_hardness(samples, label_field, hardness_field='hardness')[source]

Adds a hardness field to each sample scoring the difficulty that the specified label field observed in classifying the sample.

All classifications must have their logits attributes populated in order to use this method.

Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • label_field – the fiftyone.core.labels.Classification or fiftyone.core.labels.Classifications field to use from each sample

  • hardness_field ("hardness") – the field name to use to store the hardness value for each sample

hello.fiftyone.brain.compute_similarity(samples, patches_field=None, brain_key='img_sim', model='clip-vit-base32-torch', **kwargs)[source]

Uses embeddings to index the samples or their patches so that you can query/sort by similarity.

>>> compute_similarity(dataset, model="clip-vit-base32-torch", brain_key="img_sim")
>>> compute_similarity(dataset, patches_field="ground_truth", model="clip-vit-base32-torch", brain_key="gt_sim")
Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • patches_field (None) – a sample field defining the image patches in each sample

  • brain_key ("img_sim") – a brain key under which to store the results of this method

  • model ("clip-vit-base32-torch) – a fiftyone.core.models.Model or the name of a model from the Model Zoo

hello.fiftyone.brain.detection_mistakenness(samples, pred_field, label_field='ground_truth', mistakenness_field='mistakenness', missing_field='possible_missing', spurious_field='possible_spurious', use_logits=False, copy_missing=False)[source]

Computes the mistakenness of the labels in the specified label_field, scoring the chance that the labels are incorrect.

Mistakenness is computed based on the predictions in the pred_field, through either their confidence or logits attributes. This measure can be used to detect things like annotation errors and unusually hard samples.

This method supports both classifications and detections/polylines.

For classifications, a mistakenness_field field is populated on each sample that quantifies the likelihood that the label in the label_field of that sample is incorrect.

For detections/polylines, the mistakenness of each object in label_field is computed. Three types of mistakes are identified:

  • (Mistakes) Objects in label_field with a match in pred_field are assigned a mistakenness value in their mistakenness_field that captures the likelihood that the class label of the detection in label_field is a mistake. A mistakenness_field + "_loc" field is also populated that captures the likelihood that the detection in label_field is a mistake due to its localization (bounding box).

  • (Missing) Objects in pred_field with no matches in label_field but which are likely to be correct will have their missing_field attribute set to True. In addition, if copy_missing is True, copies of these objects are added to the ground truth label_field.

  • (Spurious) Objects in label_field with no matches in pred_field but which are likely to be incorrect will have their spurious_field attribute set to True.

In addition, for detections/polylines, the following sample-level fields are populated:

  • (Mistakes) The mistakenness_field of each sample is populated with the maximum mistakenness of the objects in label_field

  • (Missing) The missing_field of each sample is populated with the number of missing objects that were deemed missing from label_field.

  • (Spurious) The spurious_field of each sample is populated with the number of objects in label_field that were given deemed spurious.

Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • pred_field – the name of the predicted label field to use from each sample. Can be of type fiftyone.core.labels.Classification, fiftyone.core.labels.Classifications, or fiftyone.core.labels.Detections

  • label_field ("ground_truth") – the name of the “ground truth” label field that you want to test for mistakes with respect to the predictions in pred_field. Must have the same type as pred_field

  • mistakenness_field ("mistakenness") – the field name to use to store the mistakenness value for each sample

  • missing_field ("possible_missing) – the field in which to store per-sample counts of potential missing detections/polylines

  • spurious_field ("possible_spurious) – the field in which to store per-sample counts of potential spurious detections/polylines

  • use_logits (False) – whether to use logits (True) or confidence (False) to compute mistakenness. Logits typically yield better results, when they are available

  • copy_missing (False) – whether to copy predicted detections/polylines that were deemed to be missing into label_field

hello.fiftyone.brain.mistakenness_view(samples, min_mistakenness=0.5)[source]

Create a view containing the currently selected samples.

The following sample-level fields:

  • (mistakenness) The mistakenness field, store the mistakenness value.

  • (possible_missing) The missing field, store per-sample counts of potential missing.

  • (possible_spurious) The spurious field, store per-sample counts of potential spurious.

Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • min_mistakenness (0.5) – the mistakenness field minimum value

hello.fiftyone.brain.mistakenness_views(samples, pred_field='predictions', pred_filter=None, label_field='ground_truth', label_filter=None)[source]

Create a view containing the currently selected objects.

Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • pred_field ("predictions") – the label field to filter

  • pred_filter (None) – a fiftyone.core.expressions.ViewExpression

  • label_field ("ground_truth") – the label field to filter

  • label_filter (None) – a fiftyone.core.expressions.ViewExpression

hello.fiftyone.brain.patches_view(samples, field, **kwargs)[source]

Creates a view that contains one sample per object patch in the specified field of the collection.

Fields other than field and the default sample fields will not be included in the returned view. A sample_id field will be added that records the sample ID from which each patch was taken.

Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • field – the patches field, which must be of type fiftyone.core.labels.Detections or fiftyone.core.labels.Polylines

Returns:

a fiftyone.core.patches.PatchesView

hello.fiftyone.brain.to_patches(samples, field, **kwargs)[source]

Creates a view that contains one sample per object patch in the specified field of the collection.

Fields other than field and the default sample fields will not be included in the returned view. A sample_id field will be added that records the sample ID from which each patch was taken.

Parameters:

  • samples – a fiftyone.core.collections.SampleCollection

  • field – the patches field, which must be of type fiftyone.core.labels.Detections or fiftyone.core.labels.Polylines

Returns:

a fiftyone.core.patches.PatchesView