Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks

Sree Ram Kamabattula, Kumudha Musini, Babak Namazi, Ganesh Sankaranarayanan, Venkat Devarajan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Training Deep neural networks (DNNs) on noisy labeled datasets is a challenging problem, because learning on mislabeled examples deteriorates the performance of the network. As the ground truth availability is limited with real-world noisy datasets, previous papers created synthetic noisy datasets by randomly modifying the labels of training examples of clean datasets. However, no final conclusions can be derived by just using this random noise, since it excludes feature-dependent noise. Thus, it is imperative to generate feature-dependent noisy datasets that additionally provide ground truth. Therefore, we propose an intuitive approach to creating feature-dependent noisy datasets by utilizing the training predictions of DNNs on clean datasets that also retain true label information. We refer to these datasets as "Pseudo Noisy datasets". We conduct several experiments to establish that Pseudo noisy datasets resemble feature-dependent noisy datasets across different conditions. We further randomly generate synthetic noisy datasets with the same noise distribution as that of Pseudo noise (referred as "Randomized Noise") to empirically show that i) learning is easier with feature-dependent label noise compared to random noise, ii) irrespective of noise distribution, Pseudo noisy datasets mimic feature-dependent label noise and iii) current training methods are not generalizable to feature-dependent label noise. Therefore, we believe that Pseudo noisy datasets will be quite helpful to study and develop robust training methods.

Original languageEnglish (US)
Title of host publication2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2407-2413
Number of pages7
ISBN (Electronic)9781665442077
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021 - Melbourne, Australia
Duration: Oct 17 2021Oct 20 2021

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)1062-922X

Conference

Conference2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021
Country/TerritoryAustralia
CityMelbourne
Period10/17/2110/20/21

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'Generation and Analysis of Feature-Dependent Pseudo Noise for Training Deep Neural Networks'. Together they form a unique fingerprint.

Cite this