AR-Boost: Reducing overfitting by a robust data-driven regularization strategy

Baidya Nath Saha, Gautam Kunapuli, Nilanjan Ray, Joseph A Maldjian, Sriraam Natarajan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

We introduce a novel, robust data-driven regularization strategy called Adaptive Regularized Boosting (AR-Boost), motivated by a desire to reduce overfitting. We replace AdaBoost's hard margin with a regularized soft margin that trades-off between a larger margin, at the expense of misclassification errors. Minimizing this regularized exponential loss results in a boosting algorithm that relaxes the weak learning assumption further: it can use classifiers with error greater than 1/2. This enables a natural extension to multiclass boosting, and further reduces overfitting in both the binary and multiclass cases. We derive bounds for training and generalization errors, and relate them to AdaBoost. Finally, we show empirical results on benchmark data that establish the robustness of our approach and improved performance overall.

Original languageEnglish (US)
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2013, Proceedings
Pages1-16
Number of pages16
EditionPART 3
DOIs
StatePublished - Oct 31 2013
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2013 - Prague, Czech Republic
Duration: Sep 23 2013Sep 27 2013

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 3
Volume8190 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2013
CountryCzech Republic
CityPrague
Period9/23/139/27/13

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Saha, B. N., Kunapuli, G., Ray, N., Maldjian, J. A., & Natarajan, S. (2013). AR-Boost: Reducing overfitting by a robust data-driven regularization strategy. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2013, Proceedings (PART 3 ed., pp. 1-16). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8190 LNAI, No. PART 3). https://doi.org/10.1007/978-3-642-40994-3_1