Robust classification with noisy labels using Venn-Abers predictors

The advent of deep learning methods has led to impressive advances in computer vision tasks over the past decades, largely due to their ability to extract non-linear features that are well adapted to the task at hand. For supervised approaches, data labelling is essential to achieve a high level of performance, however this task can be so fastidious or even troublesome in difficult contexts (e.g., specific defect detection, unconventional data annotations, etc.) that experts can sometimes erroneously provide the wrong ground truth label. Considering classification problems, this paper addresses the issue of handling noisy labels in datasets. Specifically, we firstly detect the noisy samples of a dataset using set-valued labels, then improve their classification using Venn-Abers predictors. The obtained results reach more than 0.99 and 0.90 of accuracy for noisified MNIST and CIFAR-10 respectively with a 40% 2-class pair-flip noise ratio and 0.87 accuracy for CIFAR-10 with 10-class uniform 40% noise ratio.