Meta Unveils FACET Dataset for AI Fairness Evaluation

Highlights:

• Meta’s FACET dataset aids researchers in detecting bias in computer vision models.
• Meta asserts that FACET can identify fairness concerns within four distinct categories of computer vision models.

FACET, a benchmark dataset created to aid researchers in checking computer vision models for bias, was just made available by Meta Platforms Inc.

In addition to releasing FACET, the company is also updating its open-source DINOv2 toolkit. A group of artificial intelligence models called DINOv2 was first introduced in April with the intention of simplifying computer vision projects. Following a recent update, DINOv2 is now offered a commercial use license.

Researchers can use Meta’s new FACET dataset to determine if a computer vision model produces biased results. The business explained in a blog post how challenging it is to assess AI fairness using recent methods. According to Meta, FACET will make the task easier by giving researchers access to a sizable evaluation dataset that they can use to check a variety of computer vision models.

Researchers from Meta provided comprehensive information in the blog post: “The dataset is made up of 32,000 images containing 50,000 people, labeled by expert human annotators for demographic attributes (e.g., perceived gender presentation, perceived age group), additional physical attributes (e.g., perceived skin tone, hairstyle) and person-related classes (e.g., basketball player, doctor). FACET also contains person, hair, and clothing labels for 69,000 masks from SA-1B.”

By processing the images in FACET, researchers can verify that a computer vision model is fair. Subsequently, an analysis can be conducted to assess whether the model’s result accuracy differs among different images. Such variations in accuracy might indicate bias in the AI.

According to Meta, FACET can be used to identify fairness problems in four different kinds of computer vision models.

Researchers can use the dataset to identify bias in neural networks designed for classification or the task of grouping related images. It also facilitates the evaluation of object detection models. These models are made to recognize objects of interest in photos automatically.

FACET may audit AI systems that perform instance segmentation and visual grounding, two specialized object detection tasks. Instance segmentation is the process of visibly emphasizing objects of interest in a photograph, for instance, by drawing a frame around them. In turn, visual grounding models are neural networks that can analyze a photograph of an object described in natural language terms by a user.

Meta’s researchers wrote, “While FACET is for research evaluation purposes only and cannot be used for training, we’re releasing the dataset and a dataset explorer with the intention that FACET can become a standard fairness evaluation benchmark for computer vision models.”

Along with the launch of FACET, Meta recently changed the Apache 2.0 license on its DINOv2 series of open-source computer vision models. The software can be used by developers for both commercial and academic endeavors under the terms of the license.

The DINOv2 models from Meta are designed to derive relevant data points from images. According to Meta, the extracted data can be used to train other computer vision models. Meta claims that DINOv2 is substantially more accurate than a neural network it devised in 2021 for the same task of the previous generation.

In addition to modifying the license for DINOv2, Meta has added two new models to the toolkit recently. The first is designed to approximate the distance between the camera and the objects in a photograph. The second addition is a so-called semantic segmentation model that can segment an image and characterize each section.